FANoise: Singular Value-Adaptive Noise Modulation for Robust Multimodal Representation Learning
This work addresses the problem of robust multimodal representation learning for applications like text retrieval and multimodal understanding, but it is incremental as it builds on prior noise injection methods.
The paper tackled the challenge of learning robust and generalizable representations in multimodal tasks by proposing FANoise, a feature-adaptive noise injection strategy, which consistently improved performance across various base VLM models.
Representation learning is fundamental to modern machine learning, powering applications such as text retrieval and multimodal understanding. However, learning robust and generalizable representations remains challenging. While prior work has demonstrated that active noise injection, a form of data augmentation, can enhance encoding performance, most existing methods rely on heuristic or static noise, overlooking the dynamic nature of feature distributions during training. In this work, we systematically study the role of noise in representation learning from both gradient-based and feature distribution perspectives, using InfoNCE loss as a representative example. Focusing on multimodal representation learning, we propose FANoise, a novel feature-adaptive noise injection strategy. By leveraging the dynamics of contrastive learning, FANoise effectively mitigates the negative impacts of noise while preserving its benefits. Under this theoretically grounded framework, comprehensive experiments demonstrate that FANoise consistently improves overall performance on multimodal tasks across various base VLM models.