A Transformer-Based Contrastive Learning Approach for Few-Shot Sign Language Recognition
This addresses the challenge of recognizing diverse and new signs in production environments, though it appears incremental as it builds on existing contrastive and transformer methods.
The paper tackled the problem of few-shot sign language recognition from 2D data by proposing a contrastive transformer-based model, which achieved competitive results for unseen sign classes in one-shot or few-shot tasks.
Sign language recognition from sequences of monocular images or 2D poses is a challenging field, not only due to the difficulty to infer 3D information from 2D data, but also due to the temporal relationship between the sequences of information. Additionally, the wide variety of signs and the constant need to add new ones on production environments makes it infeasible to use traditional classification techniques. We propose a novel Contrastive Transformer-based model, which demonstrate to learn rich representations from body key points sequences, allowing better comparison between vector embedding. This allows us to apply these techniques to perform one-shot or few-shot tasks, such as classification and translation. The experiments showed that the model could generalize well and achieved competitive results for sign classes never seen in the training process.