SkinAugment: Auto-Encoding Speaker Conversions for Automatic Speech Translation
This work addresses data scarcity in speech translation, particularly for low-resource languages, by providing a method to augment training data with speaker diversity, though it is incremental as it builds on existing augmentation techniques.
The paper tackles the problem of limited training data in automatic speech translation by proposing autoencoding speaker conversion for data augmentation, which transforms audio to resemble another speaker's voice. It shows favorable results compared to SpecAugment on English→French and English→Romanian AST tasks and a low-resource ASR task, and when combined with machine-translated transcripts, outperforms a strong cascade model on English→French AST.
We propose autoencoding speaker conversion for training data augmentation in automatic speech translation. This technique directly transforms an audio sequence, resulting in audio synthesized to resemble another speaker's voice. Our method compares favorably to SpecAugment on English$\to$French and English$\to$Romanian automatic speech translation (AST) tasks as well as on a low-resource English automatic speech recognition (ASR) task. Further, in ablations, we show the benefits of both quantity and diversity in augmented data. Finally, we show that we can combine our approach with augmentation by machine-translated transcripts to obtain a competitive end-to-end AST model that outperforms a very strong cascade model on an English$\to$French AST task. Our method is sufficiently general that it can be applied to other speech generation and analysis tasks.