End-to-End Multi-speaker ASR with Independent Vector Analysis
This addresses the problem of recognizing speech in noisy, multi-speaker environments for applications like transcription and voice assistants, but it is incremental as it builds on existing IVA and ASR techniques.
The paper tackles multi-speaker automatic speech recognition by proposing an end-to-end system with an independent vector analysis frontend for joint source separation and dereverberation, achieving competitive performance with neural beamforming methods and extending to more speakers without retraining.
We develop an end-to-end system for multi-channel, multi-speaker automatic speech recognition. We propose a frontend for joint source separation and dereverberation based on the independent vector analysis (IVA) paradigm. It uses the fast and stable iterative source steering algorithm together with a neural source model. The parameters from the ASR module and the neural source model are optimized jointly from the ASR loss itself. We demonstrate competitive performance with previous systems using neural beamforming frontends. First, we explore the trade-offs when using various number of channels for training and testing. Second, we demonstrate that the proposed IVA frontend performs well on noisy data, even when trained on clean mixtures only. Furthermore, it extends without retraining to the separation of more speakers, which is demonstrated on mixtures of three and four speakers.