Forgis Research Night – Learning to Dream in EEG

Abstract

Invited lightning talk at Forgis Research Night (DARE Campus, Schlieren). I introduced LuMamba: a 4.6M-parameter EEG foundation model that pairs LUNA’s channel-unification module — learned queries that read any electrode montage into a fixed latent space — with a linear-time bidirectional Mamba backbone. The core idea is to “learn to dream”: instead of reconstructing the raw, noisy signal, we borrow the representation-learning half of LeCun’s world-model recipe and predict in latent space with LeJEPA, kept collapse-free by the SIGReg (Sketched Isotropic Gaussian) regularizer. Mixing a little masked reconstruction with LeJEPA keeps clusters and generalizes, reaching state-of-the-art Alzheimer’s detection (0.97 AUPR) — strongest on montages never seen during pretraining.

A short lightning talk on how we pretrain foundation models for biosignals — and why it pays to teach the model to dream rather than to copy.

What I covered

• The problem: EEG is a tiny (~20 µV) signal buried in noise (SNR < 1), and every dataset uses a different electrode montage.
• LuMamba: LUNA channel-unification (learned queries → fixed latent, reads any montage) + a linear-time bidirectional Mamba backbone — 4.6M parameters.
• Learning to dream: rather than reconstructing the raw signal, we predict in latent space with LeJEPA, kept collapse-free by SIGReg. A mixed reconstruction + LeJEPA objective keeps cluster structure and generalizes — state-of-the-art Alzheimer’s detection (0.97 AUPR), strongest on unseen montages.

Slides & resources

• Download the slides (PDF) →
• BioFoundation codebase →
• LUNA paper (arXiv) →
• Deep-dive blog post: LeJEPA EEG pretraining →

Photos

Thorir Mar Ingolfsson

Postdoctoral Researcher

Making AI run on microwatts: I build foundation models for EEG, ECG, and other biosignals, and compress them until they run in real time on wearable devices, not in the cloud.