Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions
This addresses the challenge of limited transcriptomic data in clinical settings for cancer patients, though it is incremental as it builds on existing multimodal fusion research.
The paper tackled the problem of multimodal AI predictions in cancer diagnosis and prognosis by generating crossmodal gene expression from histopathology, achieving state-of-the-art accuracy in predicting cancer grading and patient survival risk with high certainty and interpretability.
Emerging research has highlighted that artificial intelligence based multimodal fusion of digital pathology and transcriptomic features can improve cancer diagnosis (grading/subtyping) and prognosis (survival risk) prediction. However, such direct fusion for joint decision is impractical in real clinical settings, where histopathology is still the gold standard for diagnosis and transcriptomic tests are rarely requested, at least in the public healthcare system. With our novel diffusion based crossmodal generative AI model PathGen, we show that genomic expressions synthesized from digital histopathology jointly predicts cancer grading and patient survival risk with high accuracy (state-of-the-art performance), certainty (through conformal coverage guarantee) and interpretability (through distributed attention maps). PathGen code is available for open use by the research community through GitHub at https://github.com/Samiran-Dey/PathGen.