HEIR: Learning Graph-Based Motion Hierarchies
This provides a general, data-driven approach for motion modeling in fields like computer vision and robotics, addressing limitations of manual hierarchies, but it is incremental as it builds on existing graph-based and hierarchical methods.
The paper tackled the problem of modeling hierarchical motion structures by proposing a method that learns graph-based hierarchies directly from data, decomposing motions into parent-inherited patterns and local residuals, and it showed improved reconstruction in 1D and 2D cases and more realistic deformations in 3D scenes compared to baselines.
Hierarchical structures of motion exist across research fields, including computer vision, graphics, and robotics, where complex dynamics typically arise from coordinated interactions among simpler motion components. Existing methods to model such dynamics typically rely on manually-defined or heuristic hierarchies with fixed motion primitives, limiting their generalizability across different tasks. In this work, we propose a general hierarchical motion modeling method that learns structured, interpretable motion relationships directly from data. Our method represents observed motions using graph-based hierarchies, explicitly decomposing global absolute motions into parent-inherited patterns and local motion residuals. We formulate hierarchy inference as a differentiable graph learning problem, where vertices represent elemental motions and directed edges capture learned parent-child dependencies through graph neural networks. We evaluate our hierarchical reconstruction approach on three examples: 1D translational motion, 2D rotational motion, and dynamic 3D scene deformation via Gaussian splatting. Experimental results show that our method reconstructs the intrinsic motion hierarchy in 1D and 2D cases, and produces more realistic and interpretable deformations compared to the baseline on dynamic 3D Gaussian splatting scenes. By providing an adaptable, data-driven hierarchical modeling paradigm, our method offers a formulation applicable to a broad range of motion-centric tasks. Project Page: https://light.princeton.edu/HEIR/