Multimodal Smart Glove for Sign Language Recognition Using Deep Learning
It addresses the need for deployable sign language recognition for deaf individuals, but the approach is incremental as it combines existing sensors and deep learning methods.
The paper presents a smart glove system integrating flex sensors, IMU, and facial cues with an LSTM network for sign language recognition, achieving 95% accuracy and real-time inference via TensorFlow Lite.
Sign language recognition technologies can improve communication between deaf individuals and the broader community, but many existing systems face challenges in real-world deployment. This paper presents a deployable smart glove system for sign language recognition that integrates wearable sensing and deep learning. The glove incorporates flex sensors and an inertial measurement unit (IMU) to capture finger articulation and hand motion, while facial cues are obtained through a camera. Sensor data are transmitted via an ESP32-C6 microcontroller and processed using a long short-term memory (LSTM) network to model temporal gesture dynamics. Experimental results show that the proposed model achieves an overall recognition accuracy of approximately 95%. The trained model is further converted to TensorFlow Lite for real-time inference. This demonstrates the feasibility of the system for practical sign language translation applications.