RGMem: Renormalization Group-based Memory Evolution for Language Agent User Profile
This addresses the challenge of shallow personalized interactions and cross-session continuity for users of conversational AI systems, representing an incremental advancement in memory systems for language agents.
The paper tackles the problem of modeling long-term user states and behavioral consistency in LLM-based conversational systems, proposing RGMem, a renormalization group-inspired memory framework that organizes dialogue history across multiple scales to form dynamically-evolved user profiles, achieving a 15% improvement in user preference prediction accuracy compared to baseline methods.
Personalized and continuous interactions are the key to enhancing user experience in today's large language model (LLM)-based conversational systems, however, the finite context windows and static parametric memory make it difficult to model the cross-session long-term user states and behavioral consistency. Currently, the existing solutions to this predicament, such as retrieval-augmented generation (RAG) and explicit memory systems, primarily focus on fact-level storage and retrieval, lacking the capability to distill latent preferences and deep traits from the multi-turn dialogues, which limits the long-term and effective user modeling, directly leading to the personalized interactions remaining shallow, and hindering the cross-session continuity. To realize the long-term memory and behavioral consistency for Language Agents in LLM era, we propose a self-evolving memory framework RGMem, inspired by the ideology of classic renormalization group (RG) in physics, this framework enables to organize the dialogue history in multiple scales: it first extracts semantics and user insights from episodic fragments, then through hierarchical coarse-graining and rescaling operations, progressively forms a dynamically-evolved user profile. The core innovation of our work lies in modeling memory evolution as a multi-scale process of information compression and emergence, which accomplishes the high-level and accurate user profiles from noisy and microscopic-level interactions.