The American Sign Language Knowledge Graph: Infusing ASL Models with Linguistic Knowledge
This work addresses the need for more accessible language technologies for sign language users by enhancing ASL models with linguistic knowledge.
The authors tackled the problem of improving American Sign Language (ASL) models by introducing the ASL Knowledge Graph (ASLKG) from expert sources, achieving accuracies of 91% on isolated sign recognition, 14% on predicting semantic features of unseen signs, and 36% on classifying topics in YouTube-ASL videos.
Language models for American Sign Language (ASL) could make language technologies substantially more accessible to those who sign. To train models on tasks such as isolated sign recognition (ISR) and ASL-to-English translation, datasets provide annotated video examples of ASL signs. To facilitate the generalizability and explainability of these models, we introduce the American Sign Language Knowledge Graph (ASLKG), compiled from twelve sources of expert linguistic knowledge. We use the ASLKG to train neuro-symbolic models for 3 ASL understanding tasks, achieving accuracies of 91% on ISR, 14% for predicting the semantic features of unseen signs, and 36% for classifying the topic of Youtube-ASL videos.