HIL: Hybrid Imitation Learning of Diverse Parkour Skills from Videos
For researchers in physics-based character animation, this work addresses the challenge of composing diverse athletic skills in novel environments, though improvements over existing methods are incremental.
The paper proposes a hybrid imitation learning (HIL) framework combining motion tracking and adversarial imitation learning to enable simulated characters to perform diverse parkour skills in novel environments. The method improves motion quality and skill diversity while achieving competitive task completion compared to prior approaches.
Data-driven methods leveraging deep reinforcement learning have become the dominant paradigm for developing controllers that enable physically simulated characters to produce natural human-like behaviors. However, these data-driven methods often struggle to adapt to novel environments and compose diverse skills to perform more complex interaction tasks with the environment. To address these challenges, we propose a hybrid imitation learning (HIL) framework that combines motion tracking, for precise skill replication, with adversarial imitation learning, to enhance adaptability and skill composition, enabling robust dynamic control for highly athletic behaviors. This hybrid learning framework is implemented through parallel multi-task environments and a unified observation space, utilizing a goal-conditioned representation to facilitate knowledge-sharing across the hybrid parallel environments. We demonstrate the effectiveness of HIL on a parkour-style obstacle traversal task and a heading control task. Our framework enables a unified controller that not only preserves the naturalness of reference motion data, but also generalizes effectively to challenging new environments. Evaluations across procedurally generated tasks and baselines show that our method improves motion quality, increases skill diversity, and achieves competitive task completion compared to previous learning-based approaches. Results are best visualized through https://jiashunwang.github.io/HIL