Learning Diverse Humanoid Tasks via Synthetic Video Scenarios without Real World Data
This work addresses the high cost and limited diversity of real-world demonstrations for humanoid robot skill acquisition, offering a scalable alternative for learning multiple task variations.
The paper introduces a framework that uses generative AI to create synthetic video demonstrations of human movements from text prompts, enabling humanoid robots to learn diverse task execution styles without real-world data. Evaluated in four simulation scenarios, the robot successfully completes tasks and shows strong adaptability to motion variations.
The human-like morphology of humanoid robots grants them exceptional potential for agile and versatile motor capabilities, but it also introduces significant challenges in acquiring complex skills. Traditional Learning-from-Demonstrations methods are often constrained by the high cost of collecting real-world data, the difficulty of capturing motion-specific behaviors, and the limited diversity of demonstrations across individuals. Moreover, even for the same task, humans may execute the motion in multiple distinct ways. In this paper, we propose a new framework that leverages the power of Generative AI to convert textual prompts into realistic and diverse sequences of human body movements, enabling the robot to observe multiple variations of how a single task can be performed. These synthetic demonstrations are then used as a training resource, allowing the robot to learn a broad range of task-execution styles without requiring direct human intervention. We evaluate the proposed method across four simulation scenarios. Experimental results show that the robot not only completes the tasks successfully but also demonstrates strong adaptability to complex variations in motion.