Skill-Adaptive Ghost Instructors: Enhancing Retention and Reducing Over-Reliance in VR Piano Learning
This research offers an incremental improvement for VR motor-skill learning systems, specifically for piano training, by enhancing short-term skill retention and reducing over-reliance on external cues.
This paper addresses over-reliance on persistent cues in VR motor-skill learning by introducing a skill-adaptive ghost instructor whose opacity changes in real time based on learner performance. In a study with 30 participants learning piano in VR, the dynamic ghost instructor led to higher pitch and fingering accuracy and limited error increases compared to a static instructor.
Motor-skill learning systems in XR rely on persistent cues. However, constant cueing can induce overreliance and erode memorization and skill transfer. We introduce a skill-adaptive, dynamically transparent ghost instructor whose opacity adapts in real time to learner performance. In a first-person perspective, users observe a ghost hand executing piano fingering with either a static or a performance-adaptive transparency in a VR piano training application. We conducted a within-subjects study (N=30), where learners practiced with traditional Static (fixed-transparency) and our proposed Dynamic (performance-adaptive) modes and were tested without guidance immediately and after a 10-minute retention interval. Relative to Static, the Dynamic mode yielded higher pitch and fingering accuracy and limited error increases, with comparable timing. These findings suggest that adaptive transparency helps learners internalize fingerings more effectively, reducing dependency on external cues and improving short-term skill retention within immersive learning environments. We discuss design implications for motor-skill learning and outline directions for extending this approach to longer-term retention and more complex tasks.