A Partial Break of the Honeypots Defense to Catch Adversarial Attacks
This work exposes a vulnerability in a specific adversarial defense method, highlighting security risks for machine learning systems, though it is incremental as it targets a baseline version that was already updated.
The authors broke the baseline version of the honeypots defense for detecting adversarial attacks, reducing the detection true positive rate to 0% and AUC to 0.02 while maintaining original distortion bounds, and noted that the defense was later amended to mitigate this attack.
A recent defense proposes to inject "honeypots" into neural networks in order to detect adversarial attacks. We break the baseline version of this defense by reducing the detection true positive rate to 0\% and the detection AUC to 0.02, maintaining the original distortion bounds. The authors of the original paper have amended the defense in their CCS'20 paper to mitigate this attacks. To aid further research, we release the complete 2.5 hour keystroke-by-keystroke screen recording of our attack process at https://nicholas.carlini.com/code/ccs_honeypot_break.