Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handover
This work addresses a specific challenge in human-robot interaction by improving depth perception for hand-held transparent objects, with incremental advancements in method design.
The paper tackles the problem of inaccurate depth estimation for hand-held transparent objects, which is critical for human-robot handover, by proposing a Hand-Aware Depth Restoration method that leverages hand posture guidance and achieves better performance and generalization compared to existing methods.
Transparent objects are common in daily life, while their optical properties pose challenges for RGB-D cameras to capture accurate depth information. This issue is further amplified when these objects are hand-held, as hand occlusions further complicate depth estimation. For assistant robots, however, accurately perceiving hand-held transparent objects is critical to effective human-robot interaction. This paper presents a Hand-Aware Depth Restoration (HADR) method based on creating an implicit neural representation function from a single RGB-D image. The proposed method utilizes hand posture as an important guidance to leverage semantic and geometric information of hand-object interaction. To train and evaluate the proposed method, we create a high-fidelity synthetic dataset named TransHand-14K with a real-to-sim data generation scheme. Experiments show that our method has better performance and generalization ability compared with existing methods. We further develop a real-world human-to-robot handover system based on HADR, demonstrating its potential in human-robot interaction applications.