Value Under Ignorance in Universal Artificial Intelligence
This work addresses foundational issues in universal artificial intelligence for researchers, but it is incremental as it builds upon existing AIXI frameworks.
The paper tackles the problem of assigning utilities to interaction histories in AIXI agents when hypotheses only predict finite prefixes, by exploring imprecise probability distributions and Choquet integrals. The result is a generalization of the standard recursive value function, with an investigation into the computability level of these expected utilities.
We generalize the AIXI reinforcement learning agent to admit a wider class of utility functions. Assigning a utility to each possible interaction history forces us to confront the ambiguity that some hypotheses in the agent's belief distribution only predict a finite prefix of the history, which is sometimes interpreted as implying a chance of death equal to a quantity called the semimeasure loss. This death interpretation suggests one way to assign utilities to such history prefixes. We argue that it is as natural to view the belief distributions as imprecise probability distributions, with the semimeasure loss as total ignorance. This motivates us to consider the consequences of computing expected utilities with Choquet integrals from imprecise probability theory, including an investigation of their computability level. We recover the standard recursive value function as a special case. However, our most general expected utilities under the death interpretation cannot be characterized as such Choquet integrals.