Towards Measuring and Scoring Speaker Diarization Fairness
This addresses fairness issues in speaker diarization for users of speech processing applications, but it is incremental as it focuses on evaluation rather than solving biases.
The paper tackled the problem of evaluating fairness in speaker diarization by proposing a protocol and scoring method, identifying biases related to gender and accent when applied to a state-of-the-art method on a large dataset.
Speaker diarization, or the task of finding "who spoke and when", is now used in almost every speech processing application. Nevertheless, its fairness has not yet been evaluated because there was no protocol to study its biases one by one. In this paper we propose a protocol and a scoring method designed to evaluate speaker diarization fairness. This protocol is applied on a large dataset of spoken utterances and report the performances of speaker diarization depending on the gender, the age, the accent of the speaker and the length of the spoken sentence. Some biases induced by the gender, or the accent of the speaker were identified when we applied a state-of-the-art speaker diarization method.