Towards Relaxed Multimodal Inputs for Gait-based Parkinson's Disease Assessment
This work addresses practical challenges in deploying multimodal systems for Parkinson's disease assessment, offering a more flexible approach that could enhance clinical applications, though it is incremental in nature.
The paper tackles the problem of multimodal gait-based Parkinson's disease assessment by addressing limitations in synchronization and modality dependence during training and inference, achieving state-of-the-art performance with improvements of up to 16.48 percentage points in asynchronous settings.
Parkinson's disease assessment has garnered growing interest in recent years, particularly with the advent of sensor data and machine learning techniques. Among these, multimodal approaches have demonstrated strong performance by effectively integrating complementary information from various data sources. However, two major limitations hinder their practical application: (1) the need to synchronize all modalities during training, and (2) the dependence on all modalities during inference. To address these issues, we propose the first Parkinson's assessment system that formulates multimodal learning as a multi-objective optimization (MOO) problem. This not only allows for more flexible modality requirements during both training and inference, but also handles modality collapse issue during multimodal information fusion. In addition, to mitigate the imbalance within individual modalities, we introduce a margin-based class rebalancing strategy to enhance category learning. We conduct extensive experiments on three public datasets under both synchronous and asynchronous settings. The results show that our framework-Towards Relaxed InPuts (TRIP)-achieves state-of-the-art performance, outperforming the best baselines by 16.48, 6.89, and 11.55 percentage points in the asynchronous setting, and by 4.86 and 2.30 percentage points in the synchronous setting, highlighting its effectiveness and adaptability.