Live American Sign Language Letter Classification with Convolutional Neural Networks
This work addresses the challenge of real-time ASL letter recognition for accessibility applications, though it is incremental as it builds on existing hand detection techniques.
The researchers tackled the problem of recognizing American Sign Language letters in live video feeds by using a pre-trained hand joint detection model combined with a fully-connected neural network, which outperformed previous convolutional and transfer learning methods and generalized effectively to live applications.
This project is centered around building a neural network that is able to recognize ASL letters in images, particularly within the scope of a live video feed. Initial testing results came up short of expectations when both the convolutional network and VGG16 transfer learning approaches failed to generalize in settings of different backgrounds. The use of a pre-trained hand joint detection model was then adopted with the produced joint locations being fed into a fully-connected neural network. The results of this approach exceeded those of prior methods and generalized well to a live video feed application.