UV-Based 3D Hand-Object Reconstruction with Grasp Optimization
This work addresses the problem of accurate hand-object interaction modeling for computer vision and robotics applications, representing an incremental improvement over existing methods.
The paper tackles 3D hand shape reconstruction and hand-object grasp optimization from a single RGB image by introducing a dense UV coordinate map for contact regions and inference-time optimization, resulting in improved accuracy and vibrant textures that outperform state-of-the-art methods on datasets like Ho3D, FreiHAND, and DexYCB.
We propose a novel framework for 3D hand shape reconstruction and hand-object grasp optimization from a single RGB image. The representation of hand-object contact regions is critical for accurate reconstructions. Instead of approximating the contact regions with sparse points, as in previous works, we propose a dense representation in the form of a UV coordinate map. Furthermore, we introduce inference-time optimization to fine-tune the grasp and improve interactions between the hand and the object. Our pipeline increases hand shape reconstruction accuracy and produces a vibrant hand texture. Experiments on datasets such as Ho3D, FreiHAND, and DexYCB reveal that our proposed method outperforms the state-of-the-art.