Neural Sign Language Translation by Learning Tokenization
This work addresses the challenge of scarce annotated data for sign language translation, benefiting communication with the Deaf community, but it is incremental as it builds on existing tokenization approaches.
The paper tackled the problem of sign language translation by developing a semi-supervised tokenization method using adversarial, multitask, and transfer learning to avoid costly gloss-level annotations, achieving state-of-the-art improvements of 4 points in BLUE-4 and 5 points in ROUGE scores.
Sign Language Translation has attained considerable success recently, raising hopes for improved communication with the Deaf. A pre-processing step called tokenization improves the success of translations. Tokens can be learned from sign videos if supervised data is available. However, data annotation at the gloss level is costly, and annotated data is scarce. The paper utilizes Adversarial, Multitask, Transfer Learning to search for semi-supervised tokenization approaches without burden of additional labeling. It provides extensive experiments to compare all the methods in different settings to conduct a deeper analysis. In the case of no additional target annotation besides sentences, the proposed methodology attains 13.25 BLUE-4 and 36.28 ROUGE scores which improves the current state-of-the-art by 4 points in BLUE-4 and 5 points in ROUGE.