A Unified Model for Longitudinal Multi-Modal Multi-View Prediction with Missingness
This addresses the challenge of handling incomplete longitudinal medical records for better disease progression prediction in healthcare, though it is incremental as it builds on existing multi-modal integration methods.
The paper tackles the problem of predicting future pain and Kellgren-Lawrence grades in knee osteoarthritis by integrating longitudinal, multi-modal medical data with missing values, achieving improved results compared to specific models using the same data combinations.
Medical records often consist of different modalities, such as images, text, and tabular information. Integrating all modalities offers a holistic view of a patient's condition, while analyzing them longitudinally provides a better understanding of disease progression. However, real-world longitudinal medical records present challenges: 1) patients may lack some or all of the data for a specific timepoint, and 2) certain modalities or views might be absent for all patients during a particular period. In this work, we introduce a unified model for longitudinal multi-modal multi-view prediction with missingness. Our method allows as many timepoints as desired for input, and aims to leverage all available data, regardless of their availability. We conduct extensive experiments on the knee osteoarthritis dataset from the Osteoarthritis Initiative for pain and Kellgren-Lawrence grade prediction at a future timepoint. We demonstrate the effectiveness of our method by comparing results from our unified model to specific models that use the same modality and view combinations during training and evaluation. We also show the benefit of having extended temporal data and provide post-hoc analysis for a deeper understanding of each modality/view's importance for different tasks.