ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
This work provides a novel method for improving streaming video understanding in Video-LLMs without retraining, offering significant efficiency gains and performance improvements for real-time and long-video applications.
ObjectStream introduces a training-free framework that uses latent objects as memory anchors for streaming video understanding, improving Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception while reducing peak GPU memory and TTFT by approximately 50%, and surpassing the full-token baseline on offline long-video benchmarks while discarding 82.5% of visual tokens.
Streaming video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to token importance, temporal redundancy, or segment-level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training-free framework that treats latent objects as memory anchors for streaming video understanding. ObjectStream induces spatially coherent latent objects directly from frozen Video-LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded memory budget, without requiring external object detectors or segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing Video Large Language Models (Video-LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long-video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception, while reducing peak GPU mem-ory and TTFT by approximately 50%. On offline long-video benchmarks, it surpasses the full-token baseline while discarding 82.5% of visual tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming video memory.