A Modality-agnostic Multi-task Foundation Model for Human Brain Imaging
This addresses the problem of limited applicability in diverse clinical protocols for researchers and clinicians, though it appears incremental as it builds on existing foundation model concepts for a specific domain.
The paper tackles the challenge of generalizing learning-based approaches across uncalibrated medical imaging modalities like MRI by introducing BrainFM, a modality-agnostic multi-task foundation model for human brain imaging, which demonstrates robustness and effectiveness across five tasks and eleven datasets.
Recent learning-based approaches have made astonishing advances in calibrated medical imaging like computerized tomography (CT), yet they struggle to generalize in uncalibrated modalities -- notably magnetic resonance (MR) imaging, where performance is highly sensitive to the differences in MR contrast, resolution, and orientation. This prevents broad applicability to diverse real-world clinical protocols. Here we introduce BrainFM, a modality-agnostic, multi-task vision foundation model for human brain imaging. With the proposed "mild-to-severe" intra-subject generation and "real-synth" mix-up training strategy, BrainFM is resilient to the appearance of acquired images (e.g., modality, contrast, deformation, resolution, artifacts), and can be directly applied to five fundamental brain imaging tasks, including image synthesis for CT and T1w/T2w/FLAIR MRI, anatomy segmentation, scalp-to-cortical distance, bias field estimation, and registration. We evaluate the efficacy of BrainFM on eleven public datasets, and demonstrate its robustness and effectiveness across all tasks and input modalities. Code is available at https://github.com/jhuldr/BrainFM.