Clarification as Supervision: Reinforcement Learning for Vision-Language Interfaces
This addresses the interface mismatch for vision-language models in visual mathematical reasoning, offering a novel training approach without explicit annotations.
The paper tackled the problem of vision-language models omitting precise visual details needed for mathematical reasoning, proposing AC-RL to improve caption comprehensiveness through interaction, resulting in a 4.4-point accuracy gain on benchmarks and up to 39% reduction in clarification requests.
Recent text-only models demonstrate remarkable mathematical reasoning capabilities. Extending these to visual domains requires vision-language models to translate images into text descriptions. However, current models, trained to produce captions for human readers, often omit the precise details that reasoning systems require. This creates an interface mismatch: reasoners often fail not due to reasoning limitations but because they lack access to critical visual information. We propose Adaptive-Clarification Reinforcement Learning (AC-RL), which teaches vision models what information reasoners need through interaction. Our key insight is that clarification requests during training reveal information gaps; by penalizing success that requires clarification, we create pressure for comprehensive initial captions that enable the reasoner to solve the problem in a single pass. AC-RL improves average accuracy by 4.4 points over pretrained baselines across seven visual mathematical reasoning benchmarks, and analysis shows it would cut clarification requests by up to 39% if those were allowed. By treating clarification as a form of implicit supervision, AC-RL demonstrates that vision-language interfaces can be effectively learned through interaction alone, without requiring explicit annotations.