See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL
For practitioners of multimodal LLMs, VEPA provides a lightweight intermediate training stage that enhances visual grounding without task-specific supervision, addressing a known bottleneck in current training pipelines.
Multimodal LLMs often ignore visual evidence during inference due to weak visual grounding from caption-based pretraining. VEPA introduces a sufficiency-driven RL stage between pretraining and post-training that optimizes question-conditioned visual evidence descriptions, consistently improving performance on visually demanding benchmarks.
Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference. The prevailing training paradigm relies on large-scale caption-based pretraining for general alignment, followed by supervised fine-tuning and reinforcement learning to enable instruction following and complex reasoning. However, such pretraining provides only weak visual grounding: short, coarse captions bias models toward salient objects while neglecting fine-grained visual evidence. In this paper, we introduce Visual Evidence Pre-Alignment (VEPA), an intermediate stage between pretraining and post-training that explores a novel sufficiency-driven objective with Group Relative Policy Optimization (GRPO) to optimize question-conditioned visual evidence descriptions. Extensive experiments across diverse benchmarks show that our VEPA consistently enhances performance on visually demanding evaluations and complements standard supervised post-training. Further analyses show that the income stems from strengthened, transferable visual grounding, rather than from additional task-specific training.