A LLM Assisted Exploitation of AI-Guardian
This work highlights vulnerabilities in a recent security defense for adversarial machine learning, demonstrating the potential of LLMs to assist in breaking such systems, which is incremental but impactful for the security research community.
The paper tackled the problem of evaluating the robustness of AI-Guardian, a defense against adversarial examples, using GPT-4 to implement attacks without writing code, and found that the defense completely fails, offering no increased robustness compared to an undefended baseline.
Large language models (LLMs) are now highly capable at a diverse range of tasks. This paper studies whether or not GPT-4, one such LLM, is capable of assisting researchers in the field of adversarial machine learning. As a case study, we evaluate the robustness of AI-Guardian, a recent defense to adversarial examples published at IEEE S&P 2023, a top computer security conference. We completely break this defense: the proposed scheme does not increase robustness compared to an undefended baseline. We write none of the code to attack this model, and instead prompt GPT-4 to implement all attack algorithms following our instructions and guidance. This process was surprisingly effective and efficient, with the language model at times producing code from ambiguous instructions faster than the author of this paper could have done. We conclude by discussing (1) the warning signs present in the evaluation that suggested to us AI-Guardian would be broken, and (2) our experience with designing attacks and performing novel research using the most recent advances in language modeling.