https://thorirmar.com/tag/world-models/World ModelsWowchemy (https://wowchemy.com)en-usWed, 24 Jun 2026 16:00:00 +0200https://thorirmar.com/media/logo_hu7d0f33ed8ee956ba1f93a5db68cad78d_7097_300x300_fit_lanczos_3.pnghttps://thorirmar.com/tag/world-models/https://thorirmar.com/talk/forgis-research-night-learning-to-dream-in-eeg/Wed, 24 Jun 2026 16:00:00 +0200https://thorirmar.com/talk/forgis-research-night-learning-to-dream-in-eeg/<p>A short lightning talk on how we pretrain foundation models for biosignals — and why it pays to teach the model to <em>dream</em> rather than to copy.</p> <h2 id="what-i-covered">What I covered</h2> <ul> <li><strong>The problem:</strong> EEG is a tiny (~20 µV) signal buried in noise (SNR &lt; 1), and every dataset uses a different electrode montage.</li> <li><strong>LuMamba:</strong> LUNA channel-unification (learned queries → fixed latent, reads any montage) + a linear-time bidirectional Mamba backbone — 4.6M parameters.</li> <li><strong>Learning to dream:</strong> rather than reconstructing the raw signal, we predict in latent space with <strong>LeJEPA</strong>, kept collapse-free by SIGReg. A mixed reconstruction + LeJEPA objective keeps cluster structure <em>and</em> generalizes — state-of-the-art Alzheimer&rsquo;s detection (0.97 AUPR), strongest on unseen montages.</li> </ul> <h2 id="slides--resources">Slides &amp; resources</h2> <ul> <li><a href="uploads/forgis2026_slides.pdf">Download the slides (PDF) →</a></li> <li><a href="https://github.com/pulp-bio/biofoundation" target="_blank" rel="noopener">BioFoundation codebase →</a></li> <li><a href="https://arxiv.org/abs/2510.22257" target="_blank" rel="noopener">LUNA paper (arXiv) →</a></li> <li><a href="https://thorirmar.com/post/lejepa_eeg_pretraining/">Deep-dive blog post: LeJEPA EEG pretraining →</a></li> </ul> <h2 id="photos">Photos</h2> <figure id="figure-kicking-off--learning-to-dream-in-eeg-at-dare-campus"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="Kicking off — *Learning to Dream in EEG* at DARE Campus." srcset=" /talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_screen_huc26d848ece5fe00d01ed4dc0cb9b25cd_259367_691d303a1fc85a38c28105dfd12e9401.jpeg 400w, /talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_screen_huc26d848ece5fe00d01ed4dc0cb9b25cd_259367_54f4394ae74d436b30e263c1cb7d6a2f.jpeg 760w, /talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_screen_huc26d848ece5fe00d01ed4dc0cb9b25cd_259367_1200x1200_fit_q75_lanczos.jpeg 1200w" src="https://thorirmar.com/talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_screen_huc26d848ece5fe00d01ed4dc0cb9b25cd_259367_691d303a1fc85a38c28105dfd12e9401.jpeg" width="760" height="570" loading="lazy" data-zoomable /></div> </div><figcaption> Kicking off — <em>Learning to Dream in EEG</em> at DARE Campus. </figcaption></figure> <figure id="figure-a-full-room-for-forgis-research-night"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="A full room for Forgis Research Night." srcset=" /talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_room_hu5200938769a993a3d7ab8bbd04a244eb_450678_b0911c18fb5d90e8005cd37735a08384.jpeg 400w, /talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_room_hu5200938769a993a3d7ab8bbd04a244eb_450678_96eac57c2eb1860d35f69c3f35715057.jpeg 760w, /talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_room_hu5200938769a993a3d7ab8bbd04a244eb_450678_1200x1200_fit_q75_lanczos.jpeg 1200w" src="https://thorirmar.com/talk/forgis-research-night-learning-to-dream-in-eeg/Forgis_room_hu5200938769a993a3d7ab8bbd04a244eb_450678_b0911c18fb5d90e8005cd37735a08384.jpeg" width="760" height="570" loading="lazy" data-zoomable /></div> </div><figcaption> A full room for Forgis Research Night. </figcaption></figure>