BrainSymphony: A Transformer-Driven Fusion of fMRI Time Series and Structural Connectivity
This work addresses the need for more accessible and powerful computational neuroscience tools, though it appears incremental as it builds on existing transformer and multimodal architectures.
The authors tackled the problem of large, data-intensive foundation models in neuroimaging by introducing BrainSymphony, a lightweight, parameter-efficient model that achieves state-of-the-art performance on downstream benchmarks while being pre-trained on smaller datasets.
Existing foundation models for neuroimaging are often prohibitively large and data-intensive. We introduce BrainSymphony, a lightweight, parameter-efficient foundation model that achieves state-of-the-art performance while being pre-trained on significantly smaller public datasets. BrainSymphony's strong multimodal architecture processes functional MRI data through parallel spatial and temporal transformer streams, which are then efficiently distilled into a unified representation by a Perceiver module. Concurrently, it models structural connectivity from diffusion MRI using a novel signed graph transformer to encode the brain's anatomical structure. These powerful, modality-specific representations are then integrated via an adaptive fusion gate. Despite its compact design, our model consistently outperforms larger models on a diverse range of downstream benchmarks, including classification, prediction, and unsupervised network identification tasks. Furthermore, our model revealed novel insights into brain dynamics using attention maps on a unique external psilocybin neuroimaging dataset (pre- and post-administration). BrainSymphony establishes that architecturally-aware, multimodal models can surpass their larger counterparts, paving the way for more accessible and powerful research in computational neuroscience.