BELLS: A Framework Towards Future Proof Benchmarks for the Evaluation of LLM Safeguards
This addresses the problem of evaluating safety-critical LLM safeguards for researchers and practitioners, but it is incremental as it builds on existing benchmarks and introduces new test categories.
The paper tackles the lack of a widely recognized methodology to evaluate input-output safeguards for Large Language Models (LLMs) by introducing BELLS, a structured benchmark with three categories of tests, including an implemented next-gen architecture test using the MACHIAVELLI environment.
Input-output safeguards are used to detect anomalies in the traces produced by Large Language Models (LLMs) systems. These detectors are at the core of diverse safety-critical applications such as real-time monitoring, offline evaluation of traces, and content moderation. However, there is no widely recognized methodology to evaluate them. To fill this gap, we introduce the Benchmarks for the Evaluation of LLM Safeguards (BELLS), a structured collection of tests, organized into three categories: (1) established failure tests, based on already-existing benchmarks for well-defined failure modes, aiming to compare the performance of current input-output safeguards; (2) emerging failure tests, to measure generalization to never-seen-before failure modes and encourage the development of more general safeguards; (3) next-gen architecture tests, for more complex scaffolding (such as LLM-agents and multi-agent systems), aiming to foster the development of safeguards that could adapt to future applications for which no safeguard currently exists. Furthermore, we implement and share the first next-gen architecture test, using the MACHIAVELLI environment, along with an interactive visualization of the dataset.