What Makes an Evaluation Useful? Common Pitfalls and Best Practices
It addresses the problem of ensuring safe AI development for practitioners and researchers, but is incremental as it synthesizes existing knowledge into guidelines.
The paper tackles the lack of clear standards for evaluating AI safety risks by presenting best practices for designing useful evaluations, drawing on prior work and cybersecurity examples.
Following the rapid increase in Artificial Intelligence (AI) capabilities in recent years, the AI community has voiced concerns regarding possible safety risks. To support decision-making on the safe use and development of AI systems, there is a growing need for high-quality evaluations of dangerous model capabilities. While several attempts to provide such evaluations have been made, a clear definition of what constitutes a "good evaluation" has yet to be agreed upon. In this practitioners' perspective paper, we present a set of best practices for safety evaluations, drawing on prior work in model evaluation and illustrated through cybersecurity examples. We first discuss the steps of the initial thought process, which connects threat modeling to evaluation design. Then, we provide the characteristics and parameters that make an evaluation useful. Finally, we address additional considerations as we move from building specific evaluations to building a full and comprehensive evaluation suite.