Can LLM Agents Maintain a Persona in Discourse?
This addresses the challenge of achieving stable and interpretable personality in LLM agents for applications like education and healthcare, but it is incremental as it highlights inconsistencies rather than solving them.
The study tackled the problem of LLMs lacking consistent personality-aligned interactions in conversations by analyzing their ability to maintain assigned OCEAN personality traits in dyadic dialogues, finding that their performance varies significantly depending on models and settings.
Large Language Models (LLMs) are widely used as conversational agents, exploiting their capabilities in various sectors such as education, law, medicine, and more. However, LLMs are often subjected to context-shifting behaviour, resulting in a lack of consistent and interpretable personality-aligned interactions. Adherence to psychological traits lacks comprehensive analysis, especially in the case of dyadic (pairwise) conversations. We examine this challenge from two viewpoints, initially using two conversation agents to generate a discourse on a certain topic with an assigned personality from the OCEAN framework (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) as High/Low for each trait. This is followed by using multiple judge agents to infer the original traits assigned to explore prediction consistency, inter-model agreement, and alignment with the assigned personality. Our findings indicate that while LLMs can be guided toward personality-driven dialogue, their ability to maintain personality traits varies significantly depending on the combination of models and discourse settings. These inconsistencies emphasise the challenges in achieving stable and interpretable personality-aligned interactions in LLMs.