Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information
This work addresses a key limitation in personalization for AI agents by focusing on complex reasoning tasks, though it is incremental as it builds on existing memory mechanisms.
The paper tackles the challenge of enabling large language model-based agents to perform multi-hop complex reasoning over personalized user information, proposing a new task, dataset, and evaluation framework, and demonstrates the effectiveness of a hybrid memory method called HybridMem through experiments.
In large language model-based agents, memory serves as a critical capability for achieving personalization by storing and utilizing users' information. Although some previous studies have adopted memory to implement user personalization, they typically focus on preference alignment and simple question-answering. However, in the real world, complex tasks often require multi-hop reasoning on a large amount of user information, which poses significant challenges for current memory approaches. To address this limitation, we propose the multi-hop personalized reasoning task to explore how different memory mechanisms perform in multi-hop reasoning over personalized information. We explicitly define this task and construct a dataset along with a unified evaluation framework. Then, we implement various explicit and implicit memory methods and conduct comprehensive experiments. We evaluate their performance on this task from multiple perspectives and analyze their strengths and weaknesses. Besides, we explore hybrid approaches that combine both paradigms and propose the HybridMem method to address their limitations. We demonstrate the effectiveness of our proposed model through extensive experiments. To benefit the research community, we release this project at https://github.com/nuster1128/MPR.