Simulating realistic speech overlaps improves multi-talker ASR
This work addresses the challenge of acquiring real conversation data for multi-talker ASR, offering a domain-specific solution for speech processing.
The paper tackles the problem of training multi-talker automatic speech recognition (ASR) models by proposing an improved simulation technique for realistic speech overlaps, resulting in consistent improvements in word error rates across multiple datasets.
Multi-talker automatic speech recognition (ASR) has been studied to generate transcriptions of natural conversation including overlapping speech of multiple speakers. Due to the difficulty in acquiring real conversation data with high-quality human transcriptions, a naïve simulation of multi-talker speech by randomly mixing multiple utterances was conventionally used for model training. In this work, we propose an improved technique to simulate multi-talker overlapping speech with realistic speech overlaps, where an arbitrary pattern of speech overlaps is represented by a sequence of discrete tokens. With this representation, speech overlapping patterns can be learned from real conversations based on a statistical language model, such as N-gram, which can be then used to generate multi-talker speech for training. In our experiments, multi-talker ASR models trained with the proposed method show consistent improvement on the word error rates across multiple datasets.