Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generations
This addresses the challenge of maintaining consistent role-playing agents for applications like gaming or interactive storytelling, though it is incremental as it builds on existing RAG methods.
The paper tackles the problem of hallucination in retrieval-augmented generation (RAG) based role-playing agents when queries fall outside a character's knowledge, proposing Amadeus, a training-free framework that enhances persona consistency, and introduces CharacterRAG, a dataset with 15 characters and 450 question-answer pairs, showing effective modeling of character knowledge and attributes.
Recent advances in large language models (LLMs) have catalyzed research on role-playing agents (RPAs). However, the process of collecting character-specific utterances and continually updating model parameters to track rapidly changing persona attributes is resource-intensive. Although retrieval-augmented generation (RAG) can alleviate this problem, if a persona does not contain knowledge relevant to a given query, RAG-based RPAs are prone to hallucination, making it challenging to generate accurate responses. In this paper, we propose Amadeus, a training-free framework that can significantly enhance persona consistency even when responding to questions that lie beyond a character's knowledge. Amadeus is composed of Adaptive Context-aware Text Splitter (ACTS), Guided Selection (GS), and Attribute Extractor (AE). To facilitate effective RAG-based role-playing, ACTS partitions each character's persona into optimally sized, overlapping chunks and augments this representation with hierarchical contextual information. AE identifies a character's general attributes from the chunks retrieved by GS and uses these attributes as a final context to maintain robust persona consistency even when answering out-of-knowledge questions. To underpin the development and rigorous evaluation of RAG-based RPAs, we manually construct CharacterRAG, a role-playing dataset that consists of persona documents for 15 distinct fictional characters totaling 976K written characters, and 450 question-answer pairs. We find that our proposed method effectively models not only the knowledge possessed by characters, but also various attributes such as personality.