Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
This work addresses the need for standardized evaluation in memory methods for LLM agents, offering incremental improvements through a novel hybrid method.
The authors tackled the lack of systematic comparison of memory methods in LLM-based agents by proposing a unified framework and evaluating representative methods on two benchmarks, leading to a new memory method that outperforms state-of-the-art approaches.
Memory emerges as the core module in the large language model (LLM)-based agents for long-horizon complex tasks (e.g., multi-turn dialogue, game playing, scientific discovery), where memory can enable knowledge accumulation, iterative reasoning and self-evolution. A number of memory methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework that incorporates all the existing agent memory methods from a high-level perspective. We then extensively compare representative agent memory methods on two well-known benchmarks and examine the effectiveness of all methods, providing a thorough analysis of those methods. As a byproduct of our experimental analysis, we also design a new memory method by exploiting modules in the existing methods, which outperforms the state-of-the-art methods. Finally, based on these findings, we offer promising future research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide valuable new insights for future research.