SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting
This addresses the need for scalable and robust articulated object reconstruction in augmented/virtual reality and robotics, though it is incremental as it builds on 3D Gaussian Splatting.
The paper tackles the problem of reconstructing articulated objects from posed RGB images, introducing SplArt, a self-supervised framework that achieves state-of-the-art performance in part segmentation and articulation estimation, enabling real-time photorealistic rendering for novel viewpoints and articulations.
Reconstructing articulated objects prevalent in daily environments is crucial for applications in augmented/virtual reality and robotics. However, existing methods face scalability limitations (requiring 3D supervision or costly annotations), robustness issues (being susceptible to local optima), and rendering shortcomings (lacking speed or photorealism). We introduce SplArt, a self-supervised, category-agnostic framework that leverages 3D Gaussian Splatting (3DGS) to reconstruct articulated objects and infer kinematics from two sets of posed RGB images captured at different articulation states, enabling real-time photorealistic rendering for novel viewpoints and articulations. SplArt augments 3DGS with a differentiable mobility parameter per Gaussian, achieving refined part segmentation. A multi-stage optimization strategy is employed to progressively handle reconstruction, part segmentation, and articulation estimation, significantly enhancing robustness and accuracy. SplArt exploits geometric self-supervision, effectively addressing challenging scenarios without requiring 3D annotations or category-specific priors. Evaluations on established and newly proposed benchmarks, along with applications to real-world scenarios using a handheld RGB camera, demonstrate SplArt's state-of-the-art performance and real-world practicality. Code is publicly available at https://github.com/ripl/splart.