Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations
This work addresses the need for early warning systems in AI risk management for policymakers and researchers, but it is incremental as it builds on existing concepts of capability testing without introducing new methods.
The authors tackled the problem of tracking dangerous AI capabilities over time by developing a quantitative model to inform policy decisions, highlighting that failures in testing can lead to biased danger estimates or delayed threshold monitoring due to uncertainty in AI dynamics and lab competition.
We present a quantitative model for tracking dangerous AI capabilities over time. Our goal is to help the policy and research community visualise how dangerous capability testing can give us an early warning about approaching AI risks. We first use the model to provide a novel introduction to dangerous capability testing and how this testing can directly inform policy. Decision makers in AI labs and government often set policy that is sensitive to the estimated danger of AI systems, and may wish to set policies that condition on the crossing of a set threshold for danger. The model helps us to reason about these policy choices. We then run simulations to illustrate how we might fail to test for dangerous capabilities. To summarise, failures in dangerous capability testing may manifest in two ways: higher bias in our estimates of AI danger, or larger lags in threshold monitoring. We highlight two drivers of these failure modes: uncertainty around dynamics in AI capabilities and competition between frontier AI labs. Effective AI policy demands that we address these failure modes and their drivers. Even if the optimal targeting of resources is challenging, we show how delays in testing can harm AI policy. We offer preliminary recommendations for building an effective testing ecosystem for dangerous capabilities and advise on a research agenda.