Enhancing Multimodal Medical Image Classification using Cross-Graph Modal Contrastive Learning
This addresses the challenge of multimodal data integration for disease diagnosis, offering improved predictive capabilities, though it appears incremental as it builds on existing contrastive learning and graph-based methods.
The paper tackled the problem of medical image classification by integrating diverse non-image patient data with images, proposing a Cross-Graph Modal Contrastive Learning framework that outperformed unimodal methods in accuracy and interpretability on Parkinson's disease and melanoma datasets.
The classification of medical images is a pivotal aspect of disease diagnosis, often enhanced by deep learning techniques. However, traditional approaches typically focus on unimodal medical image data, neglecting the integration of diverse non-image patient data. This paper proposes a novel Cross-Graph Modal Contrastive Learning (CGMCL) framework for multimodal structured data from different data domains to improve medical image classification. The model effectively integrates both image and non-image data by constructing cross-modality graphs and leveraging contrastive learning to align multimodal features in a shared latent space. An inter-modality feature scaling module further optimizes the representation learning process by reducing the gap between heterogeneous modalities. The proposed approach is evaluated on two datasets: a Parkinson's disease (PD) dataset and a public melanoma dataset. Results demonstrate that CGMCL outperforms conventional unimodal methods in accuracy, interpretability, and early disease prediction. Additionally, the method shows superior performance in multi-class melanoma classification. The CGMCL framework provides valuable insights into medical image classification while offering improved disease interpretability and predictive capabilities.