← Projects

Transfer Entropy Bottleneck for fetal monitoring

PeriGen · 2025–present

The problem

During labour, clinicians watch two signals: uterine activity (UA) and fetal heart rate (FHR). A healthy fetus tolerates each contraction. A fetus in trouble shows heart-rate decelerations that follow the contractions. Visual reading of these traces is subjective, and hypoxic-ischemic encephalopathy (HIE), a neonatal brain injury, is rare. Thus, there are few labels to learn from.

The approach

The Transfer Entropy Bottleneck (TEB) is a causal variational Transformer with 4.26M parameters. It forecasts FHR with two latent models:

  • A prior pθ(zt∣f<t)p_\theta(z_t \mid f_{<t}) that sees only the past FHR f<tf_{<t}.
  • A posterior qϕ(zt∣f<t,u<t)q_\phi(z_t \mid f_{<t}, u_{<t}) that also sees the past UA u<tu_{<t}.

The KL divergence between the two measures the information that UA adds about future FHR:

TEu→f(t)  ≤  DKL ⁣(qϕ(zt∣f<t,u<t)  ∥  pθ(zt∣f<t)).\mathrm{TE}_{u \to f}(t) \;\le\; D_{\mathrm{KL}}\!\left( q_\phi(z_t \mid f_{<t}, u_{<t}) \;\middle\|\; p_\theta(z_t \mid f_{<t}) \right).

The KL term is a per-timestep upper bound on the transfer entropy TEu→f\mathrm{TE}_{u \to f}, so it gives a model-based estimate of directed coupling. Lag cross-attention then splits the coupling by contraction delay, which makes the result easier for clinicians to read.

A strictly causal scattering-transform front end feeds the model. It cuts future-sample filter energy to exactly 0 and keeps a 0.94–0.95 median correlation with two-sided features. The batched GPU PyTorch port runs at 4.5 ms per 22-minute segment, which is about 300×300\times faster than the NumPy version.

The results

The model was trained on healthy births only. Even so, it separates HIE from healthy recordings in 10 of 11 half-hour windows before delivery, on 1,959 held-out recordings (Cliff’s δ\delta up to 0.36, p=4×10−17p = 4 \times 10^{-17}).

Rigour

  • Pre-registered evaluation. 27 analyses with automatic PASS/FAIL verdicts, recording-level bootstrap confidence intervals and Holm-corrected tests over 6.45M forecast anchors.
  • Null controls. These controls showed that about 70% of the raw coupling signal came from source-availability timing alone.
  • Engineering. 846 commits in 15 months, 5,785 automated tests, and a PyTorch Lightning multi-GPU DDP framework with MLflow tracking.

The same codebase holds flow-matching and score-based diffusion latent priors, contrastive predictive coding, and a JEPA with a causal convolutional Transformer encoder. It also benchmarks RSSM, Transformer and Mamba models for long-horizon probabilistic forecasting at matched compute.

The work appears at Computing in Cardiology 2025 and 2026.