Transfer Learning for Cross-dataset Isolated Sign Language Recognition in Under-Resourced Datasets
This work addresses the challenge of limited annotated data for sign language recognition, but it is incremental as it builds on existing methods and datasets.
The study tackled the problem of sign language recognition in under-resourced datasets by evaluating transfer learning methods from Turkish sign language datasets, showing that specialized supervised approaches can outperform fine-tuning.
Sign language recognition (SLR) has recently achieved a breakthrough in performance thanks to deep neural networks trained on large annotated sign datasets. Of the many different sign languages, these annotated datasets are only available for a select few. Since acquiring gloss-level labels on sign language videos is difficult, learning by transferring knowledge from existing annotated sources is useful for recognition in under-resourced sign languages. This study provides a publicly available cross-dataset transfer learning benchmark from two existing public Turkish SLR datasets. We use a temporal graph convolution-based sign language recognition approach to evaluate five supervised transfer learning approaches and experiment with closed-set and partial-set cross-dataset transfer learning. Experiments demonstrate that improvement over finetuning based transfer learning is possible with specialized supervised transfer learning methods.