DreaMo: Articulated 3D Reconstruction From A Single Casual Video
This addresses the problem of costly and expert-dependent 3D reconstruction for applications in various domains, but it is incremental as it builds on template-free learning methods by handling low-coverage videos.
The paper tackles articulated 3D reconstruction from a single casually captured video with incomplete view coverage, proposing DreaMo which uses view-conditioned diffusion prior and tailored regularizations to achieve promising quality in novel-view rendering and shape reconstruction.
Articulated 3D reconstruction has valuable applications in various domains, yet it remains costly and demands intensive work from domain experts. Recent advancements in template-free learning methods show promising results with monocular videos. Nevertheless, these approaches necessitate a comprehensive coverage of all viewpoints of the subject in the input video, thus limiting their applicability to casually captured videos from online sources. In this work, we study articulated 3D shape reconstruction from a single and casually captured internet video, where the subject's view coverage is incomplete. We propose DreaMo that jointly performs shape reconstruction while solving the challenging low-coverage regions with view-conditioned diffusion prior and several tailored regularizations. In addition, we introduce a skeleton generation strategy to create human-interpretable skeletons from the learned neural bones and skinning weights. We conduct our study on a self-collected internet video collection characterized by incomplete view coverage. DreaMo shows promising quality in novel-view rendering, detailed articulated shape reconstruction, and skeleton generation. Extensive qualitative and quantitative studies validate the efficacy of each proposed component, and show existing methods are unable to solve correct geometry due to the incomplete view coverage.