Speaker and Language Change Detection using Wav2vec2 and Whisper
This work addresses speaker and language change detection for speech processing applications, but it is incremental as it adapts existing models rather than introducing new methods.
The paper tackled the problem of detecting speaker and language changes in speech by adapting pre-trained transformer networks like Wav2vec2 and Whisper, achieving speaker recognition equal error rates around 10% and language detection error rates of a few percent.
We investigate recent transformer networks pre-trained for automatic speech recognition for their ability to detect speaker and language changes in speech. We do this by simply adding speaker (change) or language targets to the labels. For Wav2vec2 pre-trained networks, we also investigate if the representation for the speaker change symbol can be conditioned to capture speaker identity characteristics. Using a number of constructed data sets we show that these capabilities are definitely there, with speaker recognition equal error rates of the order of 10% and language detection error rates of a few percent. We will publish the code for reproducibility.