General Incomplete Multimodal Learning via Dynamic Quality Perception
For practitioners deploying multimodal systems in real-world scenarios where data corruption is common, this work provides a unified solution to a previously fragmented problem, though the novelty is incremental.
The paper tackles the problem of multimodal learning robust to both inter-modality missing (entire modality absent) and intra-modality degradation (modality present but corrupted). The proposed GIML framework uses dynamic quality perception to handle both types of missing, achieving effective and generalizable performance across diverse datasets.
Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.