Both eyes open: Vigilant Incentives help Regulatory Markets improve AI Safety
This addresses the problem of AI safety regulation for governments and policymakers, offering a theoretical improvement but is incremental as it builds on existing Regulatory Markets proposals.
The paper tackles the challenge of designing effective regulation for rapidly advancing AI capabilities by analyzing incentive structures in Regulatory Markets, finding that 'Bounty Incentives' can fail to deter reckless behavior, while proposing 'Vigilant Incentives' to encourage innovation in safety evaluation.
In the context of rapid discoveries by leaders in AI, governments must consider how to design regulation that matches the increasing pace of new AI capabilities. Regulatory Markets for AI is a proposal designed with adaptability in mind. It involves governments setting outcome-based targets for AI companies to achieve, which they can show by purchasing services from a market of private regulators. We use an evolutionary game theory model to explore the role governments can play in building a Regulatory Market for AI systems that deters reckless behaviour. We warn that it is alarmingly easy to stumble on incentives which would prevent Regulatory Markets from achieving this goal. These 'Bounty Incentives' only reward private regulators for catching unsafe behaviour. We argue that AI companies will likely learn to tailor their behaviour to how much effort regulators invest, discouraging regulators from innovating. Instead, we recommend that governments always reward regulators, except when they find that those regulators failed to detect unsafe behaviour that they should have. These 'Vigilant Incentives' could encourage private regulators to find innovative ways to evaluate cutting-edge AI systems.