Imprecise Probabilities Meet Partial Observability: Game Semantics for Robust POMDPs
This work provides a theoretical foundation for RPOMDPs, clarifying uncertainty assumptions for researchers in AI and game theory, though it is incremental as it builds on existing robust MDP literature.
The paper tackles the problem of robust partially observable Markov decision processes (RPOMDPs) by showing that different assumptions on uncertainty sets affect optimal policies and values, and it establishes a game semantics linking RPOMDPs to partially observable stochastic games, proving the existence of a Nash equilibrium.
Partially observable Markov decision processes (POMDPs) rely on the key assumption that probability distributions are precisely known. Robust POMDPs (RPOMDPs) alleviate this concern by defining imprecise probabilities, referred to as uncertainty sets. While robust MDPs have been studied extensively, work on RPOMDPs is limited and primarily focuses on algorithmic solution methods. We expand the theoretical understanding of RPOMDPs by showing that 1) different assumptions on the uncertainty sets affect optimal policies and values; 2) RPOMDPs have a partially observable stochastic game (POSG) semantic; and 3) the same RPOMDP with different assumptions leads to semantically different POSGs and, thus, different policies and values. These novel semantics for RPOMDPs give access to results for POSGs, studied in game theory; concretely, we show the existence of a Nash equilibrium. Finally, we classify the existing RPOMDP literature using our semantics, clarifying under which uncertainty assumptions these existing works operate.