StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects
This addresses the challenge of category-level 3D reconstruction for articulating objects, which has applications in robotics and animation, but is incremental as it builds on existing canonicalization ideas.
The paper tackles the problem of reconstructing 3D models of articulating objects from unposed RGB images, achieving high reconstruction accuracy that improves with more views.
We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images. Reconstructing general articulating object categories % has important applications, but is challenging since objects can have wide variation in shape, articulation, appearance and topology. We address this by building on the idea of category-level articulation canonicalization -- mapping observations to a canonical articulation which enables correspondence-free multiview aggregation. Our end-to-end trainable neural network estimates feature-enriched canonical 3D point clouds, articulation joints, and part segmentation from one or more unposed images of an object. These intermediate estimates are used to generate a final implicit 3D reconstruction.Our approach reconstructs objects even when they are observed in different articulations in images with large baselines, and animation of reconstructed shapes. Quantitative and qualitative evaluations on different object categories show that our method is able to achieve high reconstruction accuracy, especially as more views are added.