S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval
This benchmark addresses the need for realistic evaluation of episodic memory in wearable AI assistants, highlighting a fundamental bottleneck in temporal grounding that current scaling approaches cannot overcome.
S-EMBER introduces a large-scale benchmark for streaming egocentric memory retrieval, comprising 3,141 videos (388 hours) and 9,448 QA pairs, revealing that temporal grounding precision does not improve with model scale, resolution, or frame density.
As wearable devices enable continuous first-person recording, AI assistants must reason across long time horizons to recall past experiences-a capability known as episodic memory. Current benchmarks often rely on offline evaluation with access to entire video files, failing to simulate the streaming reality of wearable intelligence. We introduce S-EMBER (Streaming Egocentric Memory Benchmark for Episodic Retrieval), a large-scale benchmark comprising 3,141 videos totaling 388 hours of organic activity captured via Ray-Ban Meta smart glasses. S-EMBER formalizes grounded streaming episodic retrieval, a paradigm shift from global offline search to causal, active recall triggered by visual events in a continuous stream. We provide 9,448 QA pairs requiring manual visual proof through precise temporal localization and supporting flexible response lengths to simulate natural human-AI interaction. Our extensive benchmarking of frontier models uncovers a localization paradox: while semantic reasoning improves with parameter scale, temporal grounding precision remains a stagnant architectural bottleneck that does not benefit from brute-force increases in model size, resolution, or frame density. S-EMBER establishes a hardware-authentic foundation for developing grounded, reliable episodic memory in the next generation of wearable AI agents.