Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVR
This work addresses out-of-character errors in role-playing agents for applications like gaming or simulations, representing an incremental improvement with novel reward designs.
The paper tackles the problem of role-playing agents lacking internal cognitive consistency by proposing Character-R1, a framework that provides verifiable reward signals for role-aware reasoning, resulting in significant performance improvements in knowledge and memory compared to existing methods.
Current role-playing agents (RPAs) are typically constructed by imitating surface-level behaviors, but this approach lacks internal cognitive consistency, often causing out-of-character errors in complex situations. To address this, we propose Character-R1, a framework designed to provide comprehensive verifiable reward signals for effective role-aware reasoning, which are missing in recent studies. Specifically, our framework comprises three core designs: (1) Cognitive Focus Reward, which enforces explicit label-based analysis of 10 character elements (e.g., worldview) to structure internal cognition; (2) Reference-Guided Reward, which utilizes overlap-based metrics with reference responses as optimization anchors to enhance exploration and performance; and (3) Character-Conditioned Reward Normalization, which adjusts reward distributions based on character categories to ensure robust optimization across heterogeneous roles. Extensive experiments demonstrate that Character-R1 significantly outperforms existing methods in knowledge, memory and others.