Commodifying Pointing in HRI: Simple and Fast Pointing Gesture Detection from RGB-D Images
This provides a practical, open-source solution for a common component in HRI systems, though it is incremental as it builds on existing detection methods.
The paper tackles the problem of detecting pointing gestures for human-robot interaction using a commodity RGB-D camera, achieving real-time performance at over 30Hz and empirically measuring absolute pointing accuracy against ground-truth motion-capture data.
We present and characterize a simple method for detecting pointing gestures suitable for human-robot interaction applications using a commodity RGB-D camera. We exploit a state-of-the-art Deep CNN-based detector to find hands and faces in RGB images, then examine the corresponding depth channel pixels to obtain full 3D pointing vectors. We test several methods of estimating the hand end-point of the pointing vector. The system runs at better than 30Hz on commodity hardware: exceeding the frame rate of typical RGB-D sensors. An estimate of the absolute pointing accuracy is found empirically by comparison with ground-truth data from a VICON motion-capture system, and the useful interaction volume established. Finally, we show an end-to-end test where a robot estimates where the pointing vector intersects the ground plane, and report the accuracy obtained. We provide source code as a ROS node, with the intention of contributing a commodity implementation of this common component in HRI systems.