Online Speaker Diarization with Relation Network
This work addresses the challenge of real-time speaker diarization for applications like meeting transcription, offering an incremental improvement by combining modules into a unified network with meta-learning.
The paper tackles the problem of online speaker diarization by proposing RenoSD, a system that integrates voice-activity-detection, embedding extraction, and speaker identity association into a single neural network using a meta-learning strategy, achieving consistent improvements over state-of-the-art baselines on AMI and CALLHOME datasets with lower latency and training data requirements.
In this paper, we propose an online speaker diarization system based on Relation Network, named RenoSD. Unlike conventional diariztion systems which consist of several independently-optimized modules, RenoSD implements voice-activity-detection (VAD), embedding extraction, and speaker identity association using a single deep neural network. The most striking feature of RenoSD is that it adopts a meta-learning strategy for speaker identity association. In particular, the relation network learns to learn a deep distance metric in a data-driven way and it can determine through a simple forward pass whether two given segments belong to the same speaker. As such, RenoSD can be performed in an online manner with low latency. Experimental results on AMI and CALLHOME datasets show that the proposed RenoSD system achieves consistent improvements over the state-of-the-art x-vector baseline. Compared with an existing online diarization system named UIS-RNN, RenoSD achieves a better performance using much fewer training data and at a lower time complexity.