Evaluating Identity Leakage in Speaker De-Identification Systems
This work addresses privacy risks for users of speaker de-identification technologies, showing that current methods are ineffective, which is incremental as it builds on existing evaluation frameworks.
The paper tackled the problem of residual identity leakage in speaker de-identification systems by introducing a benchmark to quantify it, revealing that all state-of-the-art systems leak identity information, with the highest performing system only slightly better than random guessing and the lowest achieving a 45% hit rate in top 50 candidates.
Speaker de-identification aims to conceal a speaker's identity while preserving intelligibility of the underlying speech. We introduce a benchmark that quantifies residual identity leakage with three complementary error rates: equal error rate, cumulative match characteristic hit rate, and embedding-space similarity measured via canonical correlation analysis and Procrustes analysis. Evaluation results reveal that all state-of-the-art speaker de-identification systems leak identity information. The highest performing system in our evaluation performs only slightly better than random guessing, while the lowest performing system achieves a 45% hit rate within the top 50 candidates based on CMC. These findings highlight persistent privacy risks in current speaker de-identification technologies.