Examining Identity Drift in Conversations of LLM Agents
It addresses identity consistency in AI-driven dialogue systems, particularly for long-term conversations, but is incremental as it examines an underexplored problem without proposing solutions.
This study investigated identity drift in LLM agents, finding that larger models experience greater drift, model differences are less impactful than parameter sizes, and assigning personas does not help maintain identity.
Large Language Models (LLMs) show impressive conversational abilities but sometimes show identity drift problems, where their interaction patterns or styles change over time. As the problem has not been thoroughly examined yet, this study examines identity consistency across nine LLMs. Specifically, we (1) investigate whether LLMs could maintain consistent patterns (or identity) and (2) analyze the effect of the model family, parameter sizes, and provided persona types. Our experiments involve multi-turn conversations on personal themes, analyzed in qualitative and quantitative ways. Experimental results indicate three findings. (1) Larger models experience greater identity drift. (2) Model differences exist, but their effect is not stronger than parameter sizes. (3) Assigning a persona may not help to maintain identity. We hope these three findings can help to improve persona stability in AI-driven dialogue systems, particularly in long-term conversations.