Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion
For clinicians and researchers in oncology, this work provides a more accurate and interpretable method for patient survival prediction by effectively fusing heterogeneous medical data modalities.
The paper proposes a multimodal deep learning framework for survival prediction that integrates histology, RNA-seq, and clinical data using low-rank bilinear cross-modal fusion. On the CHIMERA challenge dataset, it outperforms concatenation-based baselines and shows competitive generalization on hidden cohorts.
We propose a novel multimodal deep learning framework for patient-level survival prediction, which integrates whole-slide histology features, RNA-seq expression profiles, and clinical variables. Our architecture combines an ABMIL module~\cite{ilse2018attention} for slide-level representation with feedforward encoders for RNA and clinical data. These embeddings are then integrated through low-rank bilinear cross-modal fusion~\cite{liu2018efficient} to model conditional interactions across modalities while controlling parameter growth. The model outputs continuous risk scores that are subsequently mapped to survival times using a nonparametric calibration procedure based on the Kaplan--Meier estimator~\cite{kaplan1958nonparametric}. By decomposing multimodal reasoning into independent pairwise interactions, the proposed fusion design promotes structural interpretability and parameter efficiency compared with full tensor and hierarchical fusion strategies. Experiments on the CHIMERA challenge dataset demonstrate improved predictive performance over concatenation-based baselines and competitive generalization on hidden evaluation cohorts. These results indicate that the proposed framework is a promising approach for multimodal survival prediction in HR-NMIBC. The implementation is publicly available at https://github.com/hassancpu/ChimeraChallenge2025_Task_3.