VarArray: Array-Geometry-Agnostic Continuous Speech Separation
This addresses the speech overlap issue in meeting transcription for applications like automated transcription systems, though it is incremental as it adapts and combines existing elements.
The paper tackles the problem of continuous speech separation for natural conversation transcription by proposing VarArray, an array-geometry-agnostic neural network model that works with any number of microphones without retraining, achieving asclite-based speaker-agnostic word error rates of 17.5% and 20.4% on AMI development and evaluation sets in end-to-end settings.
Continuous speech separation using a microphone array was shown to be promising in dealing with the speech overlap problem in natural conversation transcription. This paper proposes VarArray, an array-geometry-agnostic speech separation neural network model. The proposed model is applicable to any number of microphones without retraining while leveraging the nonlinear correlation between the input channels. The proposed method adapts different elements that were proposed before separately, including transform-average-concatenate, conformer speech separation, and inter-channel phase differences, and combines them in an efficient and cohesive way. Large-scale evaluation was performed with two real meeting transcription tasks by using a fully developed transcription system requiring no prior knowledge such as reference segmentations, which allowed us to measure the impact that the continuous speech separation system could have in realistic settings. The proposed model outperformed a previous approach to array-geometry-agnostic modeling for all of the geometry configurations considered, achieving asclite-based speaker-agnostic word error rates of 17.5% and 20.4% for the AMI development and evaluation sets, respectively, in the end-to-end setting using no ground-truth segmentations.