Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models
This addresses the issue of ensuring trustworthy capability evaluations for frontier AI systems, particularly for labs and regulators, though it is incremental as it builds on existing detection methods.
The paper tackles the problem of detecting intentional underperformance (sandbagging) in language models by introducing a model-agnostic noise injection method, which improves performance on benchmarks like MMLU and enables a classifier to identify sandbagging behavior.
Capability evaluations play a critical role in ensuring the safe deployment of frontier AI systems, but this role may be undermined by intentional underperformance or ``sandbagging.'' We present a novel model-agnostic method for detecting sandbagging behavior using noise injection. Our approach is founded on the observation that introducing Gaussian noise into the weights of models either prompted or fine-tuned to sandbag can considerably improve their performance. We test this technique across a range of model sizes and multiple-choice question benchmarks (MMLU, AI2, WMDP). Our results demonstrate that noise injected sandbagging models show performance improvements compared to standard models. Leveraging this effect, we develop a classifier that consistently identifies sandbagging behavior. Our unsupervised technique can be immediately implemented by frontier labs or regulatory bodies with access to weights to improve the trustworthiness of capability evaluations.