Reasoning about Body-Parts Relations for Sign Language Recognition
This work addresses sign language recognition for accessibility applications, but it is incremental as it builds on existing sensor-based approaches.
The paper tackles sign language recognition by modeling hand movements in relation to other body parts and incorporating hand postures, resulting in improved performance compared to methods using only global hand trajectories.
Over the years, hand gesture recognition has been mostly addressed considering hand trajectories in isolation. However, in most sign languages, hand gestures are defined on a particular context (body region). We propose a pipeline to perform sign language recognition which models hand movements in the context of other parts of the body captured in the 3D space using the MS Kinect sensor. In addition, we perform sign recognition based on the different hand postures that occur during a sign. Our experiments show that considering different body parts brings improved performance when compared to other methods which only consider global hand trajectories. Finally, we demonstrate that the combination of hand postures features with hand gestures features helps to improve the prediction of a given sign.