Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline
For researchers building LLM agents, this work identifies a key design principle for memory systems that generalize across diverse scenarios, addressing a gap in current memory system evaluation.
The paper evaluates eight memory systems across five scenarios and finds that memory performance depends on giving the agent active control over storage and retrieval, leading to the development of AutoMEM, which achieves the best cross-scenario generality.
LLM agents accumulate histories that outgrow their context windows, motivating a growing literature on memory systems. Yet most existing designs are tuned to a single scenario (multi-session chat or a single trajectory format), and there is little evidence that they generalize across the heterogeneous trajectories agents encounter in deployment. We revisit eight memory systems plus an agentic harness for search problems, on five scenarios: single-turn QA, multi-session chat, agentic-trajectory QA, memory stress tests, and long-horizon agentic tasks. The harness, which self-manages flat text-file storage via tool calls, achieves the best cross-task ranking, suggesting that memory performance hinges on giving the agent active control over storage and retrieval rather than on a passive store behind a fixed pipeline. We instantiate this insight in AutoMEM, an agentic memory harness with a self-managed tool interface that achieves the best cross-scenario generality among the systems we evaluate.