Study on the temporal pooling used in deep neural networks for speaker verification
This work provides incremental insights into optimizing neural network components for speaker verification, a domain-specific task in speech processing.
The study investigated how the content of the temporal pooling layer in x-vector architectures affects training dynamics and performance in speaker verification, finding that including higher-order statistics like skewness and kurtosis influences verification performance and reveals external information beyond speaker identity.
The x-vector architecture has recently achieved state-of-the-art results on the speaker verification task. This architecture incorporates a central layer, referred to as temporal pooling, which stacks statistical parameters of the acoustic frame distribution. This work proposes to highlight the significant effect of the temporal pooling content on the training dynamics and task performance. An evaluation with different pooling layers is conducted, that is, including different statistical measures of central tendency. Notably, 3rd and 4th moment-based statistics (skewness and kurtosis) are also tested to complete the usual mean and standard-deviation parameters. Our experiments show the influence of the pooling layer content in terms of speaker verification performance, but also for several classification tasks (speaker, channel or text related), and allow to better reveal the presence of external information to the speaker identity depending on the layer content.