Reconstructing Hand-Held Objects from Monocular Video
This addresses the challenge of detailed object reconstruction for applications in robotics and AR/VR, though it is incremental by improving accuracy over prior methods.
The paper tackles the problem of reconstructing hand-held objects from monocular video by leveraging hand motion as multiple views, achieving more accurate and detailed geometry without learned priors, as validated on a new dataset with 3D ground truth.
This paper presents an approach that reconstructs a hand-held object from a monocular video. In contrast to many recent methods that directly predict object geometry by a trained network, the proposed approach does not require any learned prior about the object and is able to recover more accurate and detailed object geometry. The key idea is that the hand motion naturally provides multiple views of the object and the motion can be reliably estimated by a hand pose tracker. Then, the object geometry can be recovered by solving a multi-view reconstruction problem. We devise an implicit neural representation-based method to solve the reconstruction problem and address the issues of imprecise hand pose estimation, relative hand-object motion, and insufficient geometry optimization for small objects. We also provide a newly collected dataset with 3D ground truth to validate the proposed approach.