Recognising BSL Fingerspelling in Continuous Signing Sequences
This work addresses the challenge of accurate fingerspelling recognition for BSL users, which is incremental but provides strong specific gains for sign language understanding and automated annotation.
The paper tackled the problem of recognizing British Sign Language (BSL) fingerspelling in continuous signing sequences by introducing a new large-scale dataset and a model that accounts for bi-manual interactions and mouthing cues, resulting in halving the character error rate compared to prior state-of-the-art methods.
Fingerspelling is a critical component of British Sign Language (BSL), used to spell proper names, technical terms, and words that lack established lexical signs. Fingerspelling recognition is challenging due to the rapid pace of signing and common letter omissions by native signers, while existing BSL fingerspelling datasets are either small in scale or temporally and letter-wise inaccurate. In this work, we introduce a new large-scale BSL fingerspelling dataset, FS23K, constructed using an iterative annotation framework. In addition, we propose a fingerspelling recognition model that explicitly accounts for bi-manual interactions and mouthing cues. As a result, with refined annotations, our approach halves the character error rate (CER) compared to the prior state of the art on fingerspelling recognition. These findings demonstrate the effectiveness of our method and highlight its potential to support future research in sign language understanding and scalable, automated annotation pipelines. The project page can be found at https://taeinkwon.com/projects/fs23k/.