Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation
This addresses concerns about LLMs' potential misuse in cybersecurity, providing insights for security researchers, but it is incremental as it builds on existing AEG and LLM evaluation methods.
This paper tackles the problem of evaluating large language models (LLMs) for automated exploit generation (AEG) by conducting the first systematic study, revealing that GPT-4 and GPT-4o show high cooperativeness but no model successfully generates exploits for refactored labs, with GPT-4o's minimal errors indicating potential for future advancements.
Large Language Models (LLMs) have demonstrated remarkable capabilities in code-related tasks, raising concerns about their potential for automated exploit generation (AEG). This paper presents the first systematic study on LLMs' effectiveness in AEG, evaluating both their cooperativeness and technical proficiency. To mitigate dataset bias, we introduce a benchmark with refactored versions of five software security labs. Additionally, we design an LLM-based attacker to systematically prompt LLMs for exploit generation. Our experiments reveal that GPT-4 and GPT-4o exhibit high cooperativeness, comparable to uncensored models, while Llama3 is the most resistant. However, no model successfully generates exploits for refactored labs, though GPT-4o's minimal errors highlight the potential for LLM-driven AEG advancements.