PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning
This addresses the challenge of maintaining factual correctness while aligning with user preferences in LLMs, representing an incremental improvement in personalized AI systems.
The paper tackles the problem of balancing personalization and objectivity in LLMs by proposing PersonaDual, a framework that adaptively switches between general-purpose and personalized reasoning modes, achieving near interference-free performance and improved objective problem-solving.
As users increasingly expect LLMs to align with their preferences, personalized information becomes valuable. However, personalized information can be a double-edged sword: it can improve interaction but may compromise objectivity and factual correctness, especially when it is misaligned with the question. To alleviate this problem, we propose PersonaDual, a framework that supports both general-purpose objective reasoning and personalized reasoning in a single model, and adaptively switches modes based on context. PersonaDual is first trained with SFT to learn two reasoning patterns, and then further optimized via reinforcement learning with our proposed DualGRPO to improve mode selection. Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference, achieving near interference-free performance and better leveraging helpful personalized signals to improve objective problem-solving.