Adaptive Unimodal Regulation for Balanced Multimodal Information Acquisition
This addresses a specific bottleneck in multimodal learning for AI systems, though it appears incremental as it builds on existing methods for handling modality imbalance.
The paper tackles the problem of imbalanced information acquisition in multimodal learning during the early training stage, where dominant modalities suppress others, and proposes InfoReg to balance this process, resulting in improved performance across various datasets.
Sensory training during the early ages is vital for human development. Inspired by this cognitive phenomenon, we observe that the early training stage is also important for the multimodal learning process, where dataset information is rapidly acquired. We refer to this stage as the prime learning window. However, based on our observation, this prime learning window in multimodal learning is often dominated by information-sufficient modalities, which in turn suppresses the information acquisition of information-insufficient modalities. To address this issue, we propose Information Acquisition Regulation (InfoReg), a method designed to balance information acquisition among modalities. Specifically, InfoReg slows down the information acquisition process of information-sufficient modalities during the prime learning window, which could promote information acquisition of information-insufficient modalities. This regulation enables a more balanced learning process and improves the overall performance of the multimodal network. Experiments show that InfoReg outperforms related multimodal imbalanced methods across various datasets, achieving superior model performance. The code is available at https://github.com/GeWu-Lab/InfoReg_CVPR2025.