Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMs
This work addresses the challenge of creating more holistic character simulations in AI, which could benefit applications in storytelling or interactive agents, though it is incremental as it builds on existing persona simulation methods with a novel training approach.
The paper tackles the problem of simulating deep persona in large language models by moving beyond surface-level facts to capture a character's linguistic patterns and thought processes, using Lu Xun as a case study and showing that CharacterBot significantly outperforms baselines on adapted metrics for linguistic accuracy and opinion comprehension.
Previous approaches to persona simulation large language models (LLMs) have typically relied on learning basic biographical information, or using limited role-play dialogue datasets to capture a character's responses. However, a holistic representation of an individual goes beyond surface-level facts or conversations to deeper thoughts and thinking. In this work, we introduce CharacterBot, a model designed to replicate both the linguistic patterns and distinctive thought patterns as manifested in the textual works of a character. Using Lu Xun, a renowned Chinese writer as a case study, we propose four training tasks derived from his 17 essay collections. These include a pre-training task focused on mastering external linguistic structures and knowledge, as well as three fine-tuning tasks: multiple-choice question answering, generative question answering, and style transfer, each aligning the LLM with Lu Xun's internal ideation and writing style. To optimize learning across these tasks, we introduce a CharLoRA parameter updating mechanism, where a general linguistic style expert collaborates with other task-specific experts to better study both the language style and the understanding of deeper thoughts. We evaluate CharacterBot on three tasks for linguistic accuracy and opinion comprehension, demonstrating that it significantly outperforms the baselines on our adapted metrics. We hope this work inspires future research on deep character persona simulation LLMs while considering the importance of ethical standards.