VisualDeltas: Learning Preferences from Visual Quality Perturbations
This work addresses the problem of efficiently learning preferences for multimodal models by leveraging visual quality variations, benefiting researchers and practitioners in multimodal AI.
This paper introduces VisualDeltas, a preference-learning framework that extracts supervision from visual quality variations in multimodal data. It consistently outperforms rejection-sampling fine-tuning and improves generalization across diverse multimodal benchmarks and model scales.
We present VisualDeltas, a lightweight preference-learning framework that extracts supervision from visual quality variations in multimodal data. By leveraging the systematic impact of image quality on visual perception and reasoning, VisualDeltas induces informative preference signals without relying on human annotations or external teachers. The framework supports both label-free and label-based regimes, enabling flexible use of available supervision when present. Across diverse multimodal benchmarks and model scales, VisualDeltas consistently outperforms rejection-sampling fine-tuning and improves generalization, and extends naturally to a range of visual degradations.