Exploring a Unified Attention-Based Pooling Framework for Speaker Verification
This work addresses a domain-specific problem in speaker verification by improving pooling methods for more accurate speaker representations.
The paper tackled the suboptimal use of average pooling in speaker verification by proposing a unified attention-based pooling framework combined with multi-head attention, achieving better results than average pooling and vanilla attention on the Fisher and NIST SRE 2010 dataset.
The pooling layer is an essential component in the neural network based speaker verification. Most of the current networks in speaker verification use average pooling to derive the utterance-level speaker representations. Average pooling takes every frame as equally important, which is suboptimal since the speaker-discriminant power is different between speech segments. In this paper, we present a unified attention-based pooling framework and combine it with the multi-head attention. Experiments on the Fisher and NIST SRE 2010 dataset show that involving outputs from lower layers to compute the attention weights can outperform average pooling and achieve better results than vanilla attention method. The multi-head attention further improves the performance.