CVAIMay 26

InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward

arXiv:2605.2652048.6
Predicted impact top 5% in CV · last 90 daysOriginality Highly original
AI Analysis

This work addresses the shallow, text-centric reasoning of current VLMs by enabling long-horizon visual-textual reasoning, which is crucial for complex visual understanding tasks.

InterSketch introduces an interleaved visual-textual chain-of-thought reasoning model that dynamically generates and self-corrects visual sketches, achieving superior performance on visual reasoning benchmarks, outperforming Gemini-3-Pro.

While vision-language models (VLMs) have exhibited multi-turn visual reasoning capabilities, their reasoning trajectories remain relatively shallow and are dominated by a text-centric paradigm, limiting their applicability to complex visual challenges. In contrast, human-like thought typically involves long-horizon reasoning with an interleaved visual-textual chain-of-thought (VT-CoT). To bridge this gap, we introduce InterSketch, an interleaved reasoning model to enhance the VT-CoT capability via self-correcting and stepwise reward mechanisms. InterSketch dynamically generates intermediate visual sketches using external tools and interleaves them with textual reasoning, enabling effective perception and logical reasoning over long-horizon visual understanding tasks. Specifically, in the first cold-start stage, we propose a synthesized high-quality interleaved VT-CoT dataset and include a reflection mechanism to enable the model's capability in multi-turn interleaved reasoning and self-correction. In the subsequent reinforcement learning (RL) stage, we design a stepwise reward mechanism to mitigate the sparsity of reward signals inherent in end-only supervision over long-horizon reasoning. Extensive experiments on visual reasoning benchmarks demonstrate the effectiveness of InterSketch, even outperforming proprietary models such as Gemini-3-Pro.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes