Dual-Path Modeling for Long Recording Speech Separation in Meetings
This work addresses speech separation in real-world meeting scenarios with varying speakers, offering incremental improvements in efficiency and accuracy for applications like transcription.
The paper tackled the problem of continuous speech separation in long recordings by extending dual-path modeling with transformer layers, achieving consistent word error rate (WER) reductions on the LibriCSS dataset, including a 30% computation reduction and a 10% relative WER improvement in online processing.
The continuous speech separation (CSS) is a task to separate the speech sources from a long, partially overlapped recording, which involves a varying number of speakers. A straightforward extension of conventional utterance-level speech separation to the CSS task is to segment the long recording with a size-fixed window and process each window separately. Though effective, this extension fails to model the long dependency in speech and thus leads to sub-optimum performance. The recent proposed dual-path modeling could be a remedy to this problem, thanks to its capability in jointly modeling the cross-window dependency and the local-window processing. In this work, we further extend the dual-path modeling framework for CSS task. A transformer-based dual-path system is proposed, which integrates transform layers for global modeling. The proposed models are applied to LibriCSS, a real recorded multi-talk dataset, and consistent WER reduction can be observed in the ASR evaluation for separated speech. Also, a dual-path transformer equipped with convolutional layers is proposed. It significantly reduces the computation amount by 30% with better WER evaluation. Furthermore, the online processing dual-path models are investigated, which shows 10% relative WER reduction compared to the baseline.