alternating attention

CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attention
Alternating-attention EEG foundation model pre-trained on 20k+ hours that doubles speed and cuts memory by 6× versus standard transformers.
Alexandru Dimofte, Glenn Anta Bucagu, Thorir Mar Ingolfsson, Xiaying Wang, Andrea Cossettini, Luca Benini, Yawei Li
PDF Project DOI