A Contextual-Bandit Oversight Game with Two-Sided Informational Asymmetry
For AI safety researchers, it provides a formal framework to analyze oversight failures due to asymmetric information, though the analysis is limited to a simplified bandit setting.
The paper studies runtime human oversight of an AI agent under two-sided informational asymmetry, where the human privately knows her reward and the AI privately knows action quality. It introduces a contextual-bandit team game and characterizes a gap between team optimum and myopic human behavior, showing this gap is the price of non-credible oversight communication.
We study runtime human oversight of an AI agent when private information runs in both directions: the human privately knows her reward function, while the AI privately knows the quality of the action it proposes. This is the kind of asymmetry that arises naturally when an autonomous robot or software agent has inspected a situation its human supervisor cannot directly assess. Building on Cooperative Inverse Reinforcement Learning (CIRL) and the Oversight Game, we introduce a contextual-bandit team game with two-sided asymmetric information and a play/ask/trust/oversee interface. The bandit structure removes physical state transitions and thereby yields exact one-shot characterizations that would remain conjectural in the full POMDP setting, though the common belief remains a dynamically controlled state across rounds. We give two one-shot characterizations, a team optimum and a behaviorally natural myopic rule, whose gap is a slab of avoidable harm: a region in which the AI privately knows the proposed action is harmful and shutdown would help, yet a myopic human, trusting her prior, declines to oversee. We show this gap is the price of non-credible oversight communication, and give a partial analysis of how it resolves dynamically over repeated rounds through passive learning and active signaling with a one-period-lagged oversight response.