Decouple and Cache: KV Cache Construction for Streaming Video Understanding
For streaming video understanding models, this work addresses the underexplored challenge of cache construction in unbounded streams, enabling better generalization from short training sequences.
The paper tackles streaming video understanding with limited memory and computation, proposing DSCache, a training-free KV cache construction mechanism that decouples past and instant caches and uses position-agnostic encoding. It achieves state-of-the-art performance with an average 2.5% accuracy gain over prior methods.
Streaming video understanding requires processing unbounded video streams with limited memory and computation, posing two key challenges. First, continuously constructing new and evicting old key-value(KV) caches is required for unbounded streams. Secondly, due to the high cost of collecting and training on unbounded streams, models must learn from short sequences while generalizing to long streams. Existing streaming VideoVLLMs fail to scale to unbounded video streams or focus on cache reuse strategies, leaving the impact of cache construction underexplored. In this paper, we propose Decoupled Streaming Cache(DSCache), a training-free cache construction mechanism that adapts pretrained offline models to streaming settings. DSCache maintains a cumulative past KV cache while constructing a separate instant cache on-demand, decoupled from past caches to preserve the informativeness of recent inputs. To enable position extrapolation beyond the training length, DSCache further incorporates a position-agnostic encoding strategy, ensuring KV caches to support unseen positions and preventing position overflow. Experiments on Streaming Video QA benchmarks demonstrate DSCache's state-of-the-art performance, with an average 2.5% accuracy gains over prior methods.