Avoiding Obfuscation with Prover-Estimator Debate
This addresses a key bottleneck in scaling AI debate for accurate human supervision on complex tasks, though it is an incremental improvement over prior theoretical work.
The paper tackles the obfuscated arguments problem in recursive AI debate protocols, where dishonest debaters force honest ones to solve intractable problems, and proposes a new protocol that ensures honest debaters can win with comparable computational efficiency under stability assumptions.
Training powerful AI systems to exhibit desired behaviors hinges on the ability to provide accurate human supervision on increasingly complex tasks. A promising approach to this problem is to amplify human judgement by leveraging the power of two competing AIs in a debate about the correct solution to a given problem. Prior theoretical work has provided a complexity-theoretic formalization of AI debate, and posed the problem of designing protocols for AI debate that guarantee the correctness of human judgements for as complex a class of problems as possible. Recursive debates, in which debaters decompose a complex problem into simpler subproblems, hold promise for growing the class of problems that can be accurately judged in a debate. However, existing protocols for recursive debate run into the obfuscated arguments problem: a dishonest debater can use a computationally efficient strategy that forces an honest opponent to solve a computationally intractable problem to win. We mitigate this problem with a new recursive debate protocol that, under certain stability assumptions, ensures that an honest debater can win with a strategy requiring computational efficiency comparable to their opponent.