A Large Model for Non-invasive and Personalized Management of Breast Cancer from Multiparametric MRI
This addresses the need for non-invasive, personalized management of breast cancer patients, particularly in reducing unnecessary biopsies and predicting treatment responses, though it is incremental as it builds on existing multimodal AI approaches.
The authors tackled the problem of time-consuming and subjective interpretation of multiparametric MRI for breast cancer by developing a large mixture-of-modality-experts model (MOME), which matched senior radiologists' performance in identifying breast cancer and outperformed a junior radiologist.
Breast Magnetic Resonance Imaging (MRI) demonstrates the highest sensitivity for breast cancer detection among imaging modalities and is standard practice for high-risk women. Interpreting the multi-sequence MRI is time-consuming and prone to subjective variation. We develop a large mixture-of-modality-experts model (MOME) that integrates multiparametric MRI information within a unified structure, leveraging breast MRI scans from 5,205 female patients in China for model development and validation. MOME matches four senior radiologists' performance in identifying breast cancer and outperforms a junior radiologist. The model is able to reduce unnecessary biopsies in Breast Imaging-Reporting and Data System (BI-RADS) 4 patients, classify triple-negative breast cancer, and predict pathological complete response to neoadjuvant chemotherapy. MOME further supports inference with missing modalities and provides decision explanations by highlighting lesions and measuring modality contributions. To summarize, MOME exemplifies an accurate and robust multimodal model for noninvasive, personalized management of breast cancer patients via multiparametric MRI. Code is available at https://github.com/LLYXC/MOME/tree/main.