VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks
This addresses privacy risks for users of speech technologies by revealing critical vulnerabilities in anonymization methods, though it is incremental as it builds on existing differential privacy and membership inference concepts.
The paper tackled the problem of evaluating privacy in voice anonymization, showing that current Equal Error Rate (EER) evaluations underestimate leakage, as adversaries achieve orders-of-magnitude stronger attacks at low false-positive rates and recover attributes like gender and accent with near-perfect accuracy.
Voice anonymization aims to conceal speaker identity and attributes while preserving intelligibility, but current evaluations rely almost exclusively on Equal Error Rate (EER) that obscures whether adversaries can mount high-precision attacks. We argue that privacy should instead be evaluated in the low false-positive rate (FPR) regime, where even a small number of successful identifications constitutes a meaningful breach. To this end, we introduce VoxGuard, a framework grounded in differential privacy and membership inference that formalizes two complementary notions: User Privacy, preventing speaker re-identification, and Attribute Privacy, protecting sensitive traits such as gender and accent. Across synthetic and real datasets, we find that informed adversaries, especially those using fine-tuned models and max-similarity scoring, achieve orders-of-magnitude stronger attacks at low-FPR despite similar EER. For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization. Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation, and recommend VoxGuard as a benchmark for evaluating privacy leakage.