Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios
This addresses the problem of accurately identifying overlapping speakers in meeting recordings, which is crucial for transcription and analysis, representing a novel method for a known bottleneck.
The paper tackles overlapping speech diarization in meetings by reformulating it as a single-label prediction problem using power set encoding, which explicitly models speaker overlaps and eliminates threshold selection. The proposed SEND framework achieves state-of-the-art performance with fewer parameters and lower computational complexity in real meeting scenarios.
Overlapping speech diarization has been traditionally treated as a multi-label classification problem. In this paper, we reformulate this task as a single-label prediction problem by encoding multiple binary labels into a single label with the power set, which represents the possible combinations of target speakers. This formulation has two benefits. First, the overlaps of target speakers are explicitly modeled. Second, threshold selection is no longer needed. Through this formulation, we propose the speaker embedding-aware neural diarization (SEND) framework, where a speech encoder, a speaker encoder, two similarity scorers, and a post-processing network are jointly optimized to predict the encoded labels according to the similarities between speech features and speaker embeddings. Experimental results show that SEND has a stable learning process and can be trained on highly overlapped data without extra initialization. More importantly, our method achieves the state-of-the-art performance in real meeting scenarios with fewer model parameters and lower computational complexity.