CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attention
Abstract
CEReBrO introduces a compact EEG foundation model that tokenizes multi-channel recordings into per-channel patches and employs an alternating attention mechanism to jointly model intra-channel temporal dynamics and inter-channel spatial correlations. Compared to standard self-attention, the architecture delivers a two-fold speedup and requires six times less memory while supporting models with 3.6–85 million parameters. Pre-trained on more than 20,000 hours of publicly available EEG data with diverse channel configurations, CEReBrO sets new benchmarks for emotion and seizure detection tasks and maintains competitive performance for anomaly classification and gait prediction.
Key Highlights
- Alternating attention tokenizes EEG into per-channel patches and jointly captures temporal and spatial correlations.
- Achieves 2× speed improvement and 6× lower memory usage compared to standard self-attention architectures.
- Pre-trained on over 20,000 hours of EEG, setting new benchmarks for emotion and seizure detection tasks.
Thorir Mar Ingolfsson
Postdoctoral Researcher
I develop efficient machine learning systems for biomedical wearables that operate under extreme resource constraints. My work bridges foundation models, neural architecture design, and edge deployment to enable real-time biosignal analysis on microwatt-scale devices.