AdaFocus: Knowing When and Where to Look for Adaptive Visual Reasoning
This work provides a more efficient and accurate visual reasoning method for users of MLLMs, particularly those concerned with computational cost and precise localization.
This paper addresses the computational expense of MLLMs by proposing AdaFocus, a training-free framework for adaptive visual reasoning. AdaFocus achieves substantial performance gains and an approximate 4.0x inference speedup compared to the SOTA method ZoomEyes.
Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is computationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which adds overhead and noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose AdaFocus, a novel training-free framework designed for adaptive visual reasoning. AdaFocus follows a two-stage pipeline: a confidence-based module decides when to crop, and a semantic-guided localization module determines where to crop. This enables adaptive visual reasoning without additional training. Experimentally, AdaFocus delivers substantial performance gains while achieving approximately 4.0\times speedup inference speedup than the SOTA method ZoomEyes, representing a significant advance in both accuracy and efficiency.