OmniControl: Control Any Joint at Any Time for Human Motion Generation
This addresses the limitation in human motion generation for applications like animation or robotics by enabling more detailed control beyond just the pelvis, though it builds incrementally on existing diffusion-based models.
The authors tackled the problem of generating human motions with flexible spatial control over multiple joints at different times, achieving significant improvement over state-of-the-art methods on pelvis control and promising results for other joints.
We present a novel approach named OmniControl for incorporating flexible spatial control signals into a text-conditioned human motion generation model based on the diffusion process. Unlike previous methods that can only control the pelvis trajectory, OmniControl can incorporate flexible spatial control signals over different joints at different times with only one model. Specifically, we propose analytic spatial guidance that ensures the generated motion can tightly conform to the input control signals. At the same time, realism guidance is introduced to refine all the joints to generate more coherent motion. Both the spatial and realism guidance are essential and they are highly complementary for balancing control accuracy and motion realism. By combining them, OmniControl generates motions that are realistic, coherent, and consistent with the spatial constraints. Experiments on HumanML3D and KIT-ML datasets show that OmniControl not only achieves significant improvement over state-of-the-art methods on pelvis control but also shows promising results when incorporating the constraints over other joints.