Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora
This addresses the data bottleneck for researchers and developers in automated subtitling, though it is incremental as it builds on existing speech translation methods.
The paper tackled the lack of annotated data for speech translation subtitling by proposing an automatic method to convert existing speech translation corpora into subtitling resources, achieving similar performance to manual segmentation in comparative experiments.
Speech translation for subtitling (SubST) is the task of automatically translating speech data into well-formed subtitles by inserting subtitle breaks compliant to specific displaying guidelines. Similar to speech translation (ST), model training requires parallel data comprising audio inputs paired with their textual translations. In SubST, however, the text has to be also annotated with subtitle breaks. So far, this requirement has represented a bottleneck for system development, as confirmed by the dearth of publicly available SubST corpora. To fill this gap, we propose a method to convert existing ST corpora into SubST resources without human intervention. We build a segmenter model that automatically segments texts into proper subtitles by exploiting audio and text in a multimodal fashion, achieving high segmentation quality in zero-shot conditions. Comparative experiments with SubST systems respectively trained on manual and automatic segmentations result in similar performance, showing the effectiveness of our approach.