ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models
This addresses the problem of limited decoder capacity in EEG models for researchers and practitioners, representing a novel paradigm shift rather than an incremental improvement.
The paper tackles the limitation of large EEG models lacking decoders by introducing ECHO, a decoder-centric paradigm that reformulates EEG modeling as sequence-to-sequence learning, achieving superior generalization and outperforming state-of-the-art single-task models in multi-task settings.
Electroencephalography (EEG), with its broad range of applications, necessitates models that can generalize effectively across various tasks and datasets. Large EEG Models (LEMs) address this by pretraining encoder-centric architectures on large-scale unlabeled data to extract universal representations. While effective, these models lack decoders of comparable capacity, limiting the full utilization of the learned features. To address this issue, we introduce ECHO, a novel decoder-centric LEM paradigm that reformulates EEG modeling as sequence-to-sequence learning. ECHO captures layered relationships among signals, labels, and tasks within sequence space, while incorporating discrete support samples to construct contextual cues. This design equips ECHO with in-context learning, enabling dynamic adaptation to heterogeneous tasks without parameter updates. Extensive experiments across multiple datasets demonstrate that, even with basic model components, ECHO consistently outperforms state-of-the-art single-task LEMs in multi-task settings, showing superior generalization and adaptability.