EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
This benchmark addresses the critical need for diagnostic tools to evaluate and improve continuous episodic memory in autonomous agents, which is a foundational problem for robotics and AI.
This paper introduces EGOSTREAM, a diagnostic benchmark for evaluating streaming episodic memory in egocentric vision, featuring 2,250 questions across seven cognitive dimensions and 8,528 recall-conditioned evaluations. The benchmark reveals that current state-of-the-art memory-management mechanisms, despite comparable aggregate accuracies, exhibit distinct memory profiles and achieve a maximum accuracy of 45%, operating well below real-time.
Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools for diagnosing what models remember and for how long. We introduce \egostream, a diagnostic benchmark for streaming episodic memory evaluation in egocentric vision. \egostream organizes 2,250 curated questions along seven cognitive dimensions: detail, spatial, temporal, event, social, causal, and prospective memory. We introduce the Answer Validity Window (AVW), which specifies the temporal span an answer remains valid as the observed scene evolves. This allows us to expand the questions into 8,528 recall-conditioned evaluations, enabling controlled testing from instant to ultra-long-term recall while separating genuine model forgetting from natural world-state changes. We rigorously establish baseline performance through a unified streaming MLLM framework that compares several state-of-the-art memory-management mechanisms, covering sliding windows, attention sinks, KV-cache pruning, merging, and offloading. Experiments within a unified Qwen3-VL backbone reveal that comparable aggregate accuracies mask starkly different memory profiles. For instance, token pruning preserves fine-grained details and temporal structure significantly better than token merging, while quantized offloading rescues ultra-long-term recall. Ultimately, all mechanisms operate well below real-time (>1s per frame), and top performing methods ceil at about 45\% accuracy, exposing critical gaps in current architectures. \egostream provides the diagnostic testbed needed to close these gaps.