Unleashing the Power of Image-Tabular Self-Supervised Learning via Breaking Cross-Tabular Barriers
This work addresses the challenge of scalable multi-modal learning for medical applications by enabling better knowledge transfer across diverse data cohorts, though it is incremental in advancing existing SSL methods.
The paper tackles the problem of self-supervised learning for medical image-tabular data being limited by rigid tabular modeling, which hinders transfer across cohorts, and proposes CITab, a framework that integrates semantic cues and a prototype-guided module to improve cross-tabular learning, achieving state-of-the-art results on Alzheimer's disease diagnosis across 4,461 subjects.
Multi-modal learning integrating medical images and tabular data has significantly advanced clinical decision-making in recent years. Self-Supervised Learning (SSL) has emerged as a powerful paradigm for pretraining these models on large-scale unlabeled image-tabular data, aiming to learn discriminative representations. However, existing SSL methods for image-tabular representation learning are often confined to specific data cohorts, mainly due to their rigid tabular modeling mechanisms when modeling heterogeneous tabular data. This inter-tabular barrier hinders the multi-modal SSL methods from effectively learning transferrable medical knowledge shared across diverse cohorts. In this paper, we propose a novel SSL framework, namely CITab, designed to learn powerful multi-modal feature representations in a cross-tabular manner. We design the tabular modeling mechanism from a semantic-awareness perspective by integrating column headers as semantic cues, which facilitates transferrable knowledge learning and the scalability in utilizing multiple data sources for pretraining. Additionally, we propose a prototype-guided mixture-of-linear layer (P-MoLin) module for tabular feature specialization, empowering the model to effectively handle the heterogeneity of tabular data and explore the underlying medical concepts. We conduct comprehensive evaluations on Alzheimer's disease diagnosis task across three publicly available data cohorts containing 4,461 subjects. Experimental results demonstrate that CITab outperforms state-of-the-art approaches, paving the way for effective and scalable cross-tabular multi-modal learning.