Good and safe uses of AI Oracles
This addresses safety concerns for potentially dangerous AI systems, offering incremental improvements to existing Oracle designs.
The paper tackles the problem of designing safe AI Oracles that avoid manipulating users while still providing helpful answers, proposing two designs: a counterfactual Oracle that answers as if no one will read it, and a low-bandwidth Oracle limited in information transmission.
It is possible that powerful and potentially dangerous artificial intelligence (AI) might be developed in the future. An Oracle is a design which aims to restrain the impact of a potentially dangerous AI by restricting the agent to no actions besides answering questions. Unfortunately, most Oracles will be motivated to gain more control over the world by manipulating users through the content of their answers, and Oracles of potentially high intelligence might be very successful at this \citep{DBLP:journals/corr/AlfonsecaCACAR16}. In this paper we present two designs for Oracles which, even under pessimistic assumptions, will not manipulate their users into releasing them and yet will still be incentivised to provide their users with helpful answers. The first design is the counterfactual Oracle -- which choses its answer as if it expected nobody to ever read it. The second design is the low-bandwidth Oracle -- which is limited by the quantity of information it can transmit.