D{é}tection de locuteurs dans les s{é}ries TV
This addresses the problem of accurately identifying speakers in fictional films for applications like subtitling or content analysis, but it is incremental as it builds on existing diarization techniques with visual and dialogue constraints.
The paper tackles speaker diarization in TV series by using a two-step method that first identifies speakers locally in dialogue scenes and then clusters them globally, outperforming standard tools on the same data.
Speaker diarization of audio streams turns out to be particularly challenging when applied to fictional films, where many characters talk in various acoustic conditions (background music, sound effects, variations in intonation...). Despite this acoustic variability, such movies exhibit specific visual patterns, particularly within dialogue scenes. In this paper, we introduce a two-step method to achieve speaker diarization in TV series: speaker diarization is first performed locally within scenes visually identified as dialogues; then, the hypothesized local speakers are compared to each other during a second clustering process in order to detect recurring speakers: this second stage of clustering is subject to the constraint that the different speakers involved in the same dialogue have to be assigned to different clusters. The performances of our approach are compared to those obtained by standard speaker diarization tools applied to the same data.