STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
For researchers building video-language models, STORM offers a way to reduce inference latency and engineering complexity by internalizing temporal-visual reasoning, though it is an incremental improvement over existing externalized reasoning pipelines.
STORM introduces a two-stage framework that enables video-language models to perform spatial-temporal reasoning through internalized latent trajectories rather than explicit textual chain-of-thought, achieving improved accuracy on benchmarks like VideoMME, MVBench, TempCompass, and MMVU while reducing inference overhead compared to external tool or video-generation-based methods.
Many video reasoning tasks require tracking motion, temporal order, and evolving visual states across frames. Existing methods built on large vision-language models (LVLMs) often address this challenge by externalizing reasoning through textual chain-of-thought (CoT), keyframe selection, repeated frame reinsertion, or external tool use. While effective, such pipelines increase inference-time latency and engineering complexity, and they force temporal-visual evidence to be serialized into text or repeatedly re-encoded from frames. Inspired by the intuition that visual reasoning can occur implicitly before verbalization, we propose STORMS (Spatial-Temporal reasOning via inteRnalized Modeling), a two-stage framework that teaches LVLMs to reason through bounded continuous latent trajectories instead of explicit textual CoT. In Stage I, STORMS aligns latent tokens with thought-video representations derived from generated videos, grounding the latent states in dynamic visual evidence. In Stage II, the model is further trained with answer-only supervision, encouraging the reasoning process to be internalized without step-by-step annotations. Generated thought videos are used only during training; at inference, STORMS performs a bounded latent rollout without regenerating videos, reinserting frames, or invoking external visual tools. Experiments on VideoMME, MVBench, TempCompass, and MMVU show that STORMS improves video reasoning accuracy while substantially reducing inference overhead compared with tool or video-generation-based reasoning pipelines.