Recognition and Localisation of Pointing Gestures using a RGB-D Camera
It addresses the lack of non-verbal communication access for BVIP in team workflows, though it is incremental as it applies existing RGB-D methods to a specific domain.
The paper tackles the problem of detecting and localizing pointing gestures for blind and visually impaired people (BVIP) using an RGB-D camera, achieving success rates of up to 89.59% for a 2x3 matrix and 73.57% for a 3x4 matrix in user studies.
Non-verbal communication is part of our regular conversation, and multiple gestures are used to exchange information. Among those gestures, pointing is the most important one. If such gestures cannot be perceived by other team members, e.g. by blind and visually impaired people (BVIP), they lack important information and can hardly participate in a lively workflow. Thus, this paper describes a system for detecting such pointing gestures to provide input for suitable output modalities to BVIP. Our system employs an RGB-D camera to recognize the pointing gestures performed by the users. The system also locates the target of pointing e.g. on a common workspace. We evaluated the system by conducting a user study with 26 users. The results show that the system has a success rate of 89.59 and 79.92 % for a 2 x 3 matrix using the left and right arm respectively, and 73.57 and 68.99 % for 3 x 4 matrix using the left and right arm respectively.