CLiVR: Conversational Learning System in Virtual Reality with AI-Powered Patients
This provides a scalable, immersive supplement to standardized patient training for medical and nursing education, though it is incremental as it builds on existing VR and AI technologies.
The study tackled the resource-intensive limitations of traditional medical training simulations by developing CLiVR, a virtual reality system with AI-powered patients, which demonstrated strong user acceptance and high confidence in educational potential in an expert study with 13 medical school faculty.
Simulations constitute a fundamental component of medical and nursing education and traditionally employ standardized patients (SP) and high-fidelity manikins to develop clinical reasoning and communication skills. However, these methods require substantial resources, limiting accessibility and scalability. In this study, we introduce CLiVR, a Conversational Learning system in Virtual Reality that integrates large language models (LLMs), speech processing, and 3D avatars to simulate realistic doctor-patient interactions. Developed in Unity and deployed on the Meta Quest 3 platform, CLiVR enables trainees to engage in natural dialogue with virtual patients. Each simulation is dynamically generated from a syndrome-symptom database and enhanced with sentiment analysis to provide feedback on communication tone. Through an expert user study involving medical school faculty (n=13), we assessed usability, realism, and perceived educational impact. Results demonstrated strong user acceptance, high confidence in educational potential, and valuable feedback for improvement. CLiVR offers a scalable, immersive supplement to SP-based training.