Speaker Diarization with Region Proposal Network
This addresses the limitation of standard diarization systems that cannot handle overlapped speech, which is important for speech applications like transcription and analysis.
The paper tackles the problem of overlapped speech in speaker diarization by proposing RPNSD, a method that uses a neural network to generate speech segment proposals and compute speaker embeddings simultaneously, achieving remarkable improvements over the state-of-the-art baseline on three datasets.
Speaker diarization is an important pre-processing step for many speech applications, and it aims to solve the "who spoke when" problem. Although the standard diarization systems can achieve satisfactory results in various scenarios, they are composed of several independently-optimized modules and cannot deal with the overlapped speech. In this paper, we propose a novel speaker diarization method: Region Proposal Network based Speaker Diarization (RPNSD). In this method, a neural network generates overlapped speech segment proposals, and compute their speaker embeddings at the same time. Compared with standard diarization systems, RPNSD has a shorter pipeline and can handle the overlapped speech. Experimental results on three diarization datasets reveal that RPNSD achieves remarkable improvements over the state-of-the-art x-vector baseline.