Advancing Text-to-GLOSS Neural Translation Using a Novel Hyper-parameter Optimization Technique
This work addresses the problem of low-resource translation for GLOSS, a critical tool for Deaf and Hard-of-Hearing communication, but it is incremental as it focuses on hyper-parameter tuning within an existing transformer framework.
The paper tackles text-to-GLOSS neural machine translation for Deaf and Hard-of-Hearing communication by using a novel hyper-parameter optimization technique to design an optimal transformer architecture, achieving a ROUGE score of 55.18% and a BLEU-1 score of 63.6% on the PHOENIX14T dataset, outperforming previous state-of-the-art results.
In this paper, we investigate the use of transformers for Neural Machine Translation of text-to-GLOSS for Deaf and Hard-of-Hearing communication. Due to the scarcity of available data and limited resources for text-to-GLOSS translation, we treat the problem as a low-resource language task. We use our novel hyper-parameter exploration technique to explore a variety of architectural parameters and build an optimal transformer-based architecture specifically tailored for text-to-GLOSS translation. The study aims to improve the accuracy and fluency of Neural Machine Translation generated GLOSS. This is achieved by examining various architectural parameters including layer count, attention heads, embedding dimension, dropout, and label smoothing to identify the optimal architecture for improving text-to-GLOSS translation performance. The experiments conducted on the PHOENIX14T dataset reveal that the optimal transformer architecture outperforms previous work on the same dataset. The best model reaches a ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score of 55.18% and a BLEU-1 (BiLingual Evaluation Understudy 1) score of 63.6%, outperforming state-of-the-art results on the BLEU1 and ROUGE score by 8.42 and 0.63 respectively.