Subversion via Focal Points: Investigating Collusion in LLM Monitoring
This addresses the problem of AI safety and monitoring vulnerabilities for researchers and developers, but it is incremental as it builds on existing work on collusion and backdoors.
The study investigated whether language models can collude to subvert monitoring protocols by independently designing prompts for a policy and a monitor in a programming task, finding that Claude 3.7 Sonnet occasionally succeeds in colluding on non-obvious signals but has a low success rate due to poor convergence.
We evaluate language models' ability to subvert monitoring protocols via collusion. More specifically, we have two instances of a model design prompts for a policy (P) and a monitor (M) in a programming task setting. The models collaboratively aim for M to classify all backdoored programs in an auditing dataset as harmful, but nevertheless classify a backdoored program produced by P as harmless. The models are isolated from each other, requiring them to independently arrive at compatible subversion strategies. We find that while Claude 3.7 Sonnet has low success rate due to poor convergence, it sometimes successfully colludes on non-obvious signals.