AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
This addresses the problem of reproducibility and comparability in LLM safety research for researchers, though it is incremental as it builds on existing methods and datasets.
The authors tackled the fragmented and buggy ecosystem in LLM safety and robustness research by introducing AdversariaLLM, a toolbox that implements 12 attack algorithms, integrates 7 benchmark datasets, and provides features for reproducibility and comparability.
The rapid expansion of research on Large Language Model (LLM) safety and robustness has produced a fragmented and oftentimes buggy ecosystem of implementations, datasets, and evaluation methods. This fragmentation makes reproducibility and comparability across studies challenging, hindering meaningful progress. To address these issues, we introduce AdversariaLLM, a toolbox for conducting LLM jailbreak robustness research. Its design centers on reproducibility, correctness, and extensibility. The framework implements twelve adversarial attack algorithms, integrates seven benchmark datasets spanning harmfulness, over-refusal, and utility evaluation, and provides access to a wide range of open-weight LLMs via Hugging Face. The implementation includes advanced features for comparability and reproducibility such as compute-resource tracking, deterministic results, and distributional evaluation techniques. \name also integrates judging through the companion package JudgeZoo, which can also be used independently. Together, these components aim to establish a robust foundation for transparent, comparable, and reproducible research in LLM safety.