The Third DIHARD Diarization Challenge
This challenge provides a benchmark for the speaker diarization community, highlighting progress and remaining difficulties in making systems robust to varied recording conditions and conversational domains.
The DIHARD III challenge evaluated speaker diarization systems across 11 diverse domains and two speech activity conditions. It found marked improvements in diarization since DIHARD I, especially for two-party interactions, but noted that challenges remain for domains like web video.
DIHARD III was the third in a series of speaker diarization challenges intended to improve the robustness of diarization systems to variability in recording equipment, noise conditions, and conversational domain. Speaker diarization was evaluated under two speech activity conditions (diarization from a reference speech activity vs. diarization from scratch) and 11 diverse domains. The domains span a range of recording conditions and interaction types, including read audio-books, meeting speech, clinical interviews, web videos, and, for the first time, conversational telephone speech. A total of 30 organizations (forming 21teams) from industry and academia submitted 499 valid system outputs. The evaluation results indicate that speaker diarization has improved markedly since DIHARD I, particularly for two-party interactions, but that for many domains (e.g., web video) the problem remains far from solved.