Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scans
This addresses the lack of part-based reasoning in 3D object recognition for scene understanding, enabling robust part decomposition and object completion in real-world scans, though it is incremental as it builds on existing 3D recognition methods.
The paper tackled the problem of object part reasoning and completion in real-world 3D scans by proposing Neural Part Priors (NPPs), which learn geometric part priors from synthetic data and optimize them to fit scanned scenes, resulting in significant outperformance over state-of-the-art methods on the ScanNet dataset.
3D object recognition has seen significant advances in recent years, showing impressive performance on real-world 3D scan benchmarks, but lacking in object part reasoning, which is fundamental to higher-level scene understanding such as inter-object similarities or object functionality. Thus, we propose to leverage large-scale synthetic datasets of 3D shapes annotated with part information to learn Neural Part Priors (NPPs), optimizable spaces characterizing geometric part priors. Crucially, we can optimize over the learned part priors in order to fit to real-world scanned 3D scenes at test time, enabling robust part decomposition of the real objects in these scenes that also estimates the complete geometry of the object while fitting accurately to the observed real geometry. Moreover, this enables global optimization over geometrically similar detected objects in a scene, which often share strong geometric commonalities, enabling scene-consistent part decompositions. Experiments on the ScanNet dataset demonstrate that NPPs significantly outperforms state of the art in part decomposition and object completion in real-world scenes.