Stability-driven Contact Reconstruction From Monocular Color Images
This work addresses the problem of generalizing contact reconstruction to personalized hands and diverse objects for applications in robotics and human-computer interaction, though it is incremental by building on prior optimization approaches.
The paper tackles the challenge of reconstructing hand-object contact from monocular images by using a physical stability criterion in simulation to optimize contact patterns, achieving improved accuracy and stability compared to existing methods.
Physical contact provides additional constraints for hand-object state reconstruction as well as a basis for further understanding of interaction affordances. Estimating these severely occluded regions from monocular images presents a considerable challenge. Existing methods optimize the hand-object contact driven by distance threshold or prior from contact-labeled datasets. However, due to the number of subjects and objects involved in these indoor datasets being limited, the learned contact patterns could not be generalized easily. Our key idea is to reconstruct the contact pattern directly from monocular images, and then utilize the physical stability criterion in the simulation to optimize it. This criterion is defined by the resultant forces and contact distribution computed by the physics engine.Compared to existing solutions, our framework can be adapted to more personalized hands and diverse object shapes. Furthermore, an interaction dataset with extra physical attributes is created to verify the sim-to-real consistency of our methods. Through comprehensive evaluations, hand-object contact can be reconstructed with both accuracy and stability by the proposed framework.