Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints
This work addresses a domain-specific problem for vision-language model users by providing an incremental improvement in visual reasoning efficiency.
The paper tackles the problem of detailed visual reasoning in vision-language models under resource constraints by training smaller-scale models with Group Relative Policy Optimization to use external tools like zoom, achieving better performance on some visual question-answering tasks compared to similarly-sized baselines.
Despite tremendous recent advances in large model reasoning ability, vision-language models (VLMs) still struggle with detailed visual reasoning, especially when compute resources are limited. To address this challenge, we draw inspiration from methods like Deepseek-r1 for VLMs and train smaller-scale models with Group Relative Policy Optimization (GRPO) to use external tools such as zoom. The greatest benefit is obtained with a combination of GRPO learning, a simple reward structure, a simplified tool-calling interface, allocating additional tokens to the result of the tool call, and a training data mix that over-represents visually difficult examples. Compared to similarly-sized baseline models, our method achieves better performance on some visual question-answering (VQA) tasks, thanks to the detailed visual information gathered from the external tool.