Persona Switch: Mixing Distinct Perspectives in Decoding Time
This incremental improvement addresses reliability issues in persona-based prompting for language model users.
The paper tackled the inconsistency of role-play prompting in language models by proposing Persona Switch, a decoding method that dynamically combines zero-shot and role-play prompting based on output confidence, resulting in up to 5.13% accuracy improvement.
Role-play prompting is known to steer the behavior of language models by injecting a persona into the prompt, improving their zero-shot reasoning capabilities. However, such improvements are inconsistent across different tasks or instances. This inconsistency suggests that zero-shot and role-play prompting may offer complementary strengths rather than one being universally superior. Building on this insight, we propose Persona Switch, a novel decoding method that dynamically combines the benefits of both prompting strategies. Our method proceeds step-by-step, selecting the better output between zero-shot and role-play prompting at each step by comparing their output confidence, as measured by the logit gap. Experiments with widely-used LLMs demonstrate that Persona Switch consistently outperforms competitive baselines, achieving up to 5.13% accuracy improvement. Furthermore, we show that output confidence serves as an informative measure for selecting the more reliable output.