Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysis
It addresses security risks like information leakage and jailbreak attacks for organizations using LLMs, but is incremental as it focuses on comparative analysis and recommendations.
This report compares open-source vulnerability scanners for conversational LLMs, evaluating Garak, Giskard, PyRIT, and CyberSecEval, and finds significant reliability issues in detecting attacks, while contributing a preliminary labelled dataset.
This report presents a comparative analysis of open-source vulnerability scanners for conversational large language models (LLMs). As LLMs become integral to various applications, they also present potential attack surfaces, exposed to security risks such as information leakage and jailbreak attacks. Our study evaluates prominent scanners - Garak, Giskard, PyRIT, and CyberSecEval - that adapt red-teaming practices to expose these vulnerabilities. We detail the distinctive features and practical use of these scanners, outline unifying principles of their design and perform quantitative evaluations to compare them. These evaluations uncover significant reliability issues in detecting successful attacks, highlighting a fundamental gap for future development. Additionally, we contribute a preliminary labelled dataset, which serves as an initial step to bridge this gap. Based on the above, we provide strategic recommendations to assist organizations choose the most suitable scanner for their red-teaming needs, accounting for customizability, test suite comprehensiveness, and industry-specific use cases.