Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models
For researchers working on multimodal large language models, this work addresses the instability of visual latent reasoning by diagnosing and mitigating a feature-space mismatch, though the gains are incremental.
The paper identifies a feature-space mismatch in visual latent reasoning for MLLMs, where decoder hidden states used as latent inputs occupy a different norm regime than input embeddings, causing instability. The proposed GAP paradigm aligns visual latents at feature, context, and capacity levels, achieving the best mean aggregate perception and reasoning performance on Qwen2.5-VL 7B among supervised variants.
Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume~\citep{xie2025mhc,li2026siamesenorm,team2026attention}. This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose \textbf{GAP}, a \textbf{G}ranular \textbf{A}lignment \textbf{P}aradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacity-guided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.