DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension
It addresses 3D reconstruction for articulated objects without needing 3D scans, enabling broader applications across categories like different animal species, though it builds on existing DensePose concepts.
The paper tackles monocular 3D reconstruction of articulated objects like humans and animals, introducing DensePose 3D, which learns reconstructions in a weakly supervised manner from 2D image annotations only, showing significant improvements over state-of-the-art baselines on synthetic and real data.
We tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstruction methods that use parametric models such as SMPL pre-trained on a large dataset of 3D object scans. Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species. The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object. The decomposition of the object into parts is regularized by expressing part assignments as a combination of the smooth eigenfunctions of the Laplace-Beltrami operator. We show significant improvements compared to state-of-the-art non-rigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals.