CodeBrain: Towards Decoupled Interpretability and Multi-Scale Architecture for EEG Foundation Model
This addresses scalability and interpretability issues in EEG analysis for neuroscience applications, representing a novel method rather than an incremental improvement.
The paper tackles the problem of EEG foundation models producing uninterpretable and weakly discriminative representations by introducing CodeBrain, a two-stage model that decouples signals and uses a multi-scale architecture, achieving strong generalization across 8 downstream tasks and 10 datasets with distribution shifts.
Electroencephalography (EEG) provides real-time insights into brain activity and supports diverse applications in neuroscience. While EEG foundation models (EFMs) have emerged to address the scalability issues of task-specific models, current approaches still yield clinically uninterpretable and weakly discriminative representations, inefficiently capture global dependencies, and neglect important local neural events. We present CodeBrain, a two-stage EFM designed to fill this gap. In the first stage, we introduce the TFDual-Tokenizer, which decouples heterogeneous temporal and frequency EEG signals into discrete tokens, quadratically expanding the representation space to enhance discriminative power and offering domain-specific interpretability by suggesting potential links to neural events and spectral rhythms. In the second stage, we propose the multi-scale EEGSSM architecture, which combines structured global convolution with sliding window attention to efficiently capture both sparse long-range and local dependencies, reflecting the brain's small-world topology. Pretrained on the largest public EEG corpus, CodeBrain achieves strong generalization across 8 downstream tasks and 10 datasets under distribution shifts, supported by comprehensive ablations, scaling-law analyses, and interpretability evaluations. Both code and pretraining weights will be released in the future version.