Sign Language Recognition using Parallel Bidirectional Reservoir Computing
This provides a cost-effective solution for real-time sign language recognition, benefiting deaf and hearing communities, but it is incremental as it adapts existing reservoir computing techniques to a specific domain.
The paper tackled the problem of sign language recognition on edge devices by proposing a lightweight system using parallel bidirectional reservoir computing with MediaPipe, achieving top-1 accuracy of 60.85% and reducing training time to 18.67 seconds compared to over 55 minutes for deep learning methods.
Sign language recognition (SLR) facilitates communication between deaf and hearing communities. Deep learning based SLR models are commonly used but require extensive computational resources, making them unsuitable for deployment on edge devices. To address these limitations, we propose a lightweight SLR system that combines parallel bidirectional reservoir computing (PBRC) with MediaPipe. MediaPipe enables real-time hand tracking and precise extraction of hand joint coordinates, which serve as input features for the PBRC architecture. The proposed PBRC architecture consists of two echo state network (ESN) based bidirectional reservoir computing (BRC) modules arranged in parallel to capture temporal dependencies, thereby creating a rich feature representation for classification. We trained our PBRC-based SLR system on the Word-Level American Sign Language (WLASL) video dataset, achieving top-1, top-5, and top-10 accuracies of 60.85%, 85.86%, and 91.74%, respectively. Training time was significantly reduced to 18.67 seconds due to the intrinsic properties of reservoir computing, compared to over 55 minutes for deep learning based methods such as Bi-GRU. This approach offers a lightweight, cost-effective solution for real-time SLR on edge devices.