Sign Segmentation with Changepoint-Modulated Pseudo-Labelling
This addresses the challenge of limited annotation for sign language segmentation, enabling better analysis of unlabelled signing footage, though it is incremental as it builds on existing adaptation techniques.
The paper tackles the problem of finding temporal boundaries between signs in continuous sign language by proposing a source-free domain adaptation method, achieving state-of-the-art performance on datasets like BSL-1K and RWTH-PHOENIX-Weather 2014.
The objective of this work is to find temporal boundaries between signs in continuous sign language. Motivated by the paucity of annotation available for this task, we propose a simple yet effective algorithm to improve segmentation performance on unlabelled signing footage from a domain of interest. We make the following contributions: (1) We motivate and introduce the task of source-free domain adaptation for sign language segmentation, in which labelled source data is available for an initial training phase, but is not available during adaptation. (2) We propose the Changepoint-Modulated Pseudo-Labelling (CMPL) algorithm to leverage cues from abrupt changes in motion-sensitive feature space to improve pseudo-labelling quality for adaptation. (3) We showcase the effectiveness of our approach for category-agnostic sign segmentation, transferring from the BSLCORPUS to the BSL-1K and RWTH-PHOENIX-Weather 2014 datasets, where we outperform the prior state of the art.