Sign Language Conversation Interpretation Using Wearable Sensors and Machine Learning
This work addresses communication barriers for the 1.57 billion people with hearing loss, though it is a proof-of-concept with incremental improvements in sensor-based recognition.
The paper tackled the problem of automatic sign language recognition for people with hearing loss by developing a system using wearable flex sensors and machine learning, achieving up to 99% accuracy on a selected set of American Sign Language dynamic words.
The count of people suffering from various levels of hearing loss reached 1.57 billion in 2019. This huge number tends to suffer on many personal and professional levels and strictly needs to be included with the rest of society healthily. This paper presents a proof of concept of an automatic sign language recognition system based on data obtained using a wearable device of 3 flex sensors. The system is designed to interpret a selected set of American Sign Language (ASL) dynamic words by collecting data in sequences of the performed signs and using machine learning methods. The built models achieved high-quality performances, such as Random Forest with 99% accuracy, Support Vector Machine (SVM) with 99%, and two K-Nearest Neighbor (KNN) models with 98%. This indicates many possible paths toward the development of a full-scale system.