Programmatic Concept Learning for Human Motion Description and Synthesis
This work addresses the challenge of data-efficient human motion analysis and generation for applications in computer vision and robotics, though it is incremental in building on existing motion representation methods.
The paper tackles the problem of representing and synthesizing human motion by introducing Programmatic Motion Concepts, a hierarchical representation that enables description, editing, and controlled video synthesis, achieving improved performance over baselines, particularly with limited data.
We introduce Programmatic Motion Concepts, a hierarchical motion representation for human actions that captures both low-level motion and high-level description as motion concepts. This representation enables human motion description, interactive editing, and controlled synthesis of novel video sequences within a single framework. We present an architecture that learns this concept representation from paired video and action sequences in a semi-supervised manner. The compactness of our representation also allows us to present a low-resource training recipe for data-efficient learning. By outperforming established baselines, especially in the small data regime, we demonstrate the efficiency and effectiveness of our framework for multiple applications.