TikArt: Aperture-Guided Observation for Fine-Grained Visual Reasoning via Reinforcement Learning
This work addresses the problem of lost visual details in multimodal reasoning for AI researchers, offering an incremental improvement through a novel agent-based approach.
The paper tackles fine-grained visual reasoning in multimodal large language models by introducing TikArt, an aperture-guided agent that uses reinforcement learning to optimize region-of-interest actions like zooming and segmentation, resulting in consistent performance gains on benchmarks such as V*, HR-Bench-4K/8K, and MME-RealWorld-Lite.
We address fine-grained visual reasoning in multimodal large language models (MLLMs), where key evidence may reside in tiny objects, cluttered regions, or subtle markings that are lost under a single global image encoding. We introduce TikArt (Thinking Aperture), an aperture-guided agent that casts multi-step vision-language reasoning as a decision process over regions of interest. TikArt follows a Think-Aperture-Observe loop, alternating between language generation and two aperture actions: Zoom extracts rectangular crops, while Segment invokes SAM2 to obtain mask-based crops for irregular targets. After every action, the model must produce an explicit observation, turning local visual cues into persistent linguistic memory. Built on Qwen3-VL-8B, TikArt optimizes its reasoning policy with AGRPO, a GRPO-style reinforcement learning algorithm with a two-stage curriculum: it warms up segmentation actions and then jointly optimizes visual math, fine-grained VQA, and segmentation, using rewards that couple task success with purposeful aperture use. Experiments on V*, HR-Bench-4K/8K, MME-RealWorld-Lite, MMStar, RefCOCO, and ReasonSeg show consistent gains over the backbone and yield interpretable aperture trajectories for high-resolution reasoning.