Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management
This work addresses the challenge of managing external memory in LLM systems for developers and researchers, but it is incremental as it provides a reframing rather than a new algorithm.
The paper tackles the problem of memory management in large language model systems, which currently relies on hand-designed heuristics, by proposing a decision-theoretic framework called DAM that reframes it as a sequential decision-making problem under uncertainty to improve long-term utility and risk assessment.
External memory is a key component of modern large language model (LLM) systems, enabling long-term interaction and personalization. Despite its importance, memory management is still largely driven by hand-designed heuristics, offering little insight into the long-term and uncertain consequences of memory decisions. In practice, choices about what to read or write shape future retrieval and downstream behavior in ways that are difficult to anticipate. We argue that memory management should be viewed as a sequential decision-making problem under uncertainty, where the utility of memory is delayed and dependent on future interactions. To this end, we propose DAM (Decision-theoretic Agent Memory), a decision-theoretic framework that decomposes memory management into immediate information access and hierarchical storage maintenance. Within this architecture, candidate operations are evaluated via value functions and uncertainty estimators, enabling an aggregate policy to arbitrate decisions based on estimated long-term utility and risk. Our contribution is not a new algorithm, but a principled reframing that clarifies the limitations of heuristic approaches and provides a foundation for future research on uncertainty-aware memory systems.