SAPIEN: Affective Virtual Agents Powered by Large Language Models
This demo paper presents a platform for creating customizable virtual agents with broad potential applications, but it is incremental as it builds on existing LLM and virtual agent technologies.
The authors introduced SAPIEN, a platform for high-fidelity virtual agents powered by large language models that can hold open-domain conversations in 13 languages and display emotions through facial expressions and voice, with applications ranging from entertainment to mental health and education.
In this demo paper, we introduce SAPIEN, a platform for high-fidelity virtual agents driven by large language models that can hold open domain conversations with users in 13 different languages, and display emotions through facial expressions and voice. The platform allows users to customize their virtual agent's personality, background, and conversation premise, thus providing a rich, immersive interaction experience. Furthermore, after the virtual meeting, the user can choose to get the conversation analyzed and receive actionable feedback on their communication skills. This paper illustrates an overview of the platform and discusses the various application domains of this technology, ranging from entertainment to mental health, communication training, language learning, education, healthcare, and beyond. Additionally, we consider the ethical implications of such realistic virtual agent representations and the potential challenges in ensuring responsible use.