Why Did the x-Vector System Miss a Target Speaker? Impact of Acoustic Mismatch Upon Target Score on VoxCeleb Data
This work addresses interpretability and robustness issues in speaker verification systems, which is incremental as it applies existing statistical modeling to analyze a known bottleneck in a specific domain.
The study investigated why x-vector speaker verification systems fail to recognize target speakers by modeling the impact of acoustic mismatch on detection scores, finding that mismatches in F0 mean and formant frequencies are key factors causing false rejections on VoxCeleb data.
Modern automatic speaker verification (ASV) relies heavily on machine learning implemented through deep neural networks. It can be difficult to interpret the output of these black boxes. In line with interpretative machine learning, we model the dependency of ASV detection score upon acoustic mismatch of the enrollment and test utterances. We aim to identify mismatch factors that explain target speaker misses (false rejections). We use distance in the first- and second-order statistics of selected acoustic features as the predictors in a linear mixed effects model, while a standard Kaldi x-vector system forms our ASV black-box. Our results on the VoxCeleb data reveal the most prominent mismatch factor to be in F0 mean, followed by mismatches associated with formant frequencies. Our findings indicate that x-vector systems lack robustness to intra-speaker variations.