The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection
For developers and users of audio deepfake detection systems, the paper reveals a critical vulnerability in provenance watermarking that undermines detector reliability, but offers a simple fix.
The paper identifies that watermarking synthetic speech creates a spurious 'watermark => fake' shortcut for detectors, causing three failures: generalization degradation, strip-to-evade, and mark-to-frame. In a white-box experiment, mark-to-frame lifts Equal Error Rate from 16% to 75%; retraining with watermark on both classes fixes the issue.
Provenance watermarking is increasingly treated as a safeguard for synthetic speech, whether built directly into speech-generation models such as Chatterbox, provided through dedicated techniques such as AudioSeal, or deployed by commercial platforms such as ElevenLabs. We identify a previously uncharacterized liability: when synthetic speech is watermarked and human speech is not, detectors trained alongside latch onto the watermark as a spurious "watermark => fake" shortcut. This single feature yields three coupled failures: generalization degradation (model performance deteriorates on unseen data), strip-to-evade (a watermarked fake escapes once unwatermarked), and mark-to-frame (watermarking a real voice flags it as fake). In a controlled white-box experiment, a watermark-trained detector shows all three (for example, mark-to-frame lifts Equal Error Rate from 16% to 75%). In a black-box test of a commercial API, we show that adding a watermark to real speech disguises it as fake. However, this shortcut is fixable: retraining with the watermark on both classes decorrelates it and restores clean behavior. We release experiment data as a paired clean-versus-watermarked corpus (WASP).