Heterogeneous Graph Learning for Multi-modal Medical Data Analysis
This addresses the challenge of integrating diverse medical data types to improve clinical decision-making, representing an incremental advancement in multi-modal fusion methods.
The paper tackles the problem of effectively fusing multi-modal medical data (e.g., images and non-image features) for more accurate clinical decisions by proposing HetMed, a heterogeneous graph learning framework, which demonstrates superiority in experiments on real-world datasets.
Routine clinical visits of a patient produce not only image data, but also non-image data containing clinical information regarding the patient, i.e., medical data is multi-modal in nature. Such heterogeneous modalities offer different and complementary perspectives on the same patient, resulting in more accurate clinical decisions when they are properly combined. However, despite its significance, how to effectively fuse the multi-modal medical data into a unified framework has received relatively little attention. In this paper, we propose an effective graph-based framework called HetMed (Heterogeneous Graph Learning for Multi-modal Medical Data Analysis) for fusing the multi-modal medical data. Specifically, we construct a multiplex network that incorporates multiple types of non-image features of patients to capture the complex relationship between patients in a systematic way, which leads to more accurate clinical decisions. Extensive experiments on various real-world datasets demonstrate the superiority and practicality of HetMed. The source code for HetMed is available at https://github.com/Sein-Kim/Multimodal-Medical.