Instance-Agnostic Geometry and Contact Dynamics Learning
This addresses the need for more flexible contact learning in robotics and simulation by eliminating dependency on motion capture and shape priors, though it appears incremental as an integration of existing methods.
The paper tackles the problem of learning object geometry and contact dynamics from RGBD video without requiring shape priors, by fusing vision and dynamics systems in a cyclic training pipeline, and demonstrates improved tracking performance for rigid convex objects.
This work presents an instance-agnostic learning framework that fuses vision with dynamics to simultaneously learn shape, pose trajectories, and physical properties via the use of geometry as a shared representation. Unlike many contact learning approaches that assume motion capture input and a known shape prior for the collision model, our proposed framework learns an object's geometric and dynamic properties from RGBD video, without requiring either category-level or instance-level shape priors. We integrate a vision system, BundleSDF, with a dynamics system, ContactNets, and propose a cyclic training pipeline to use the output from the dynamics module to refine the poses and the geometry from the vision module, using perspective reprojection. Experiments demonstrate our framework's ability to learn the geometry and dynamics of rigid and convex objects and improve upon the current tracking framework.