Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design
It addresses methodological gaps in evaluating LLM-based cybersecurity attacks for researchers, but is incremental as it reviews existing work without proposing new methods.
This paper analyzed benchmarking practices for evaluating LLM-driven offensive security tools by reviewing 19 research papers, finding that methodology quality depends on testbeds and metrics, and provided recommendations for improving future research.
Large Language Models (LLMs) have emerged as a powerful approach for driving offensive penetration-testing tooling. Due to the opaque nature of LLMs, empirical methods are typically used to analyze their efficacy. The quality of this analysis is highly dependent on the chosen testbed, captured metrics and analysis methods employed. This paper analyzes the methodology and benchmarking practices used for evaluating Large Language Model (LLM)-driven attacks, focusing on offensive uses of LLMs in cybersecurity. We review 19 research papers detailing 18 prototypes and their respective testbeds. We detail our findings and provide actionable recommendations for future research, emphasizing the importance of extending existing testbeds, creating baselines, and including comprehensive metrics and qualitative analysis. We also note the distinction between security research and practice, suggesting that CTF-based challenges may not fully represent real-world penetration testing scenarios.