Are disentangled representations all you need to build speaker anonymization systems?
This work addresses privacy concerns in speech data collection by enhancing speaker anonymization, but it is incremental as it builds on existing disentanglement and voice conversion methods.
The paper tackled the problem of imperfect disentanglement in speaker anonymization systems, which leads to privacy leakage, and found that using vector quantization to remove speaker information from acoustic models improves identity concealment while preserving speech recognition utility.
Speech signals contain a lot of sensitive information, such as the speaker's identity, which raises privacy concerns when speech data get collected. Speaker anonymization aims to transform a speech signal to remove the source speaker's identity while leaving the spoken content unchanged. Current methods perform the transformation by relying on content/speaker disentanglement and voice conversion. Usually, an acoustic model from an automatic speech recognition system extracts the content representation while an x-vector system extracts the speaker representation. Prior work has shown that the extracted features are not perfectly disentangled. This paper tackles how to improve features disentanglement, and thus the converted anonymized speech. We propose enhancing the disentanglement by removing speaker information from the acoustic model using vector quantization. Evaluation done using the VoicePrivacy 2022 toolkit showed that vector quantization helps conceal the original speaker identity while maintaining utility for speech recognition.