Generalized Feedback Loop for Joint Hand-Object Pose Estimation
This work addresses the challenge of accurately estimating poses for hand-object interactions in robotics or AR/VR, though it is incremental as it builds on existing CNN-based methods with a feedback mechanism.
The paper tackles the problem of 3D hand and object pose estimation from depth images by introducing a feedback loop to correct CNN predictions, achieving real-time performance and outperforming state-of-the-art methods for joint hand-object pose estimation.
We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.