Designing an Effective Metric Learning Pipeline for Speaker Diarization
This work addresses the challenge of improving speaker diarization accuracy for applications like transcription and meeting analysis, but it is incremental as it focuses on optimizing existing pipeline components rather than introducing a new method.
The paper tackles the problem of building robust speaker diarization systems by emphasizing the design of the metric learning pipeline, including loss functions and sampling strategies, and proposes a fine-grained validation process to evaluate generalization across languages and speaker counts. It provides empirical insights and recommendations based on these studies.
State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained $i-$vectors to representations learned via different sequence modeling architectures (e.g. 1D-CNNs, LSTMs, attention models), while adopting off-the-shelf metric learning solutions. In this paper, we argue that, regardless of the feature extractor, it is crucial to carefully design a metric learning pipeline, namely the loss function, the sampling strategy and the discrimnative margin parameter, for building robust diarization systems. Furthermore, we propose to adopt a fine-grained validation process to obtain a comprehensive evaluation of the generalization power of metric learning pipelines. To this end, we measure diarization performance across different language speakers, and variations in the number of speakers in a recording. Using empirical studies, we provide interesting insights into the effectiveness of different design choices and make recommendations.