Forecasting Rare Language Model Behaviors
This addresses a critical safety problem for AI developers deploying language models at scale, though it is an incremental improvement on existing evaluation methods.
The paper tackles the problem of standard language model evaluations failing to capture rare but dangerous behaviors that emerge only at deployment scale, and introduces a forecasting method that predicts such risks across up to three orders of magnitude more queries than tested, enabling proactive patching before large-scale deployment.
Standard language model evaluations can fail to capture risks that emerge only at deployment scale. For example, a model may produce safe responses during a small-scale beta test, yet reveal dangerous information when processing billions of requests at deployment. To remedy this, we introduce a method to forecast potential risks across orders of magnitude more queries than we test during evaluation. We make forecasts by studying each query's elicitation probability -- the probability the query produces a target behavior -- and demonstrate that the largest observed elicitation probabilities predictably scale with the number of queries. We find that our forecasts can predict the emergence of diverse undesirable behaviors -- such as assisting users with dangerous chemical synthesis or taking power-seeking actions -- across up to three orders of magnitude of query volume. Our work enables model developers to proactively anticipate and patch rare failures before they manifest during large-scale deployments.