MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
This addresses the challenge of consistent 3D motion estimation for robotics or computer vision, but it is incremental as it builds on existing methods like spectral synchronization.
The paper tackles the problem of multi-body motion segmentation and rigid registration from multiple 3D point clouds, achieving strong generalizability across different object categories with effective results on various datasets.
We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds. The two non-trivial challenges posed by this multi-scan multibody setting that we investigate are: (i) guaranteeing correspondence and segmentation consistency across multiple input point clouds capturing different spatial arrangements of bodies or body parts; and (ii) obtaining robust motion-based rigid body segmentation applicable to novel object categories. We propose an approach to address these issues that incorporates spectral synchronization into an iterative deep declarative network, so as to simultaneously recover consistent correspondences as well as motion segmentation. At the same time, by explicitly disentangling the correspondence and motion segmentation estimation modules, we achieve strong generalizability across different object categories. Our extensive evaluations demonstrate that our method is effective on various datasets ranging from rigid parts in articulated objects to individually moving objects in a 3D scene, be it single-view or full point clouds.