Towards Using Fully Observable Policies for POMDPs
This work addresses a specific challenge in POMDPs for applications like game theory or robotics, but it appears incremental as it builds on existing fully observable methods.
The paper tackles the problem of solving Partially Observable Markov Decision Processes (POMDPs) with multimodal belief by proposing an approach that uses policies from the fully observable version, resulting in outperforming policies that ignore multiple modes on a benchmark based on Reconnaissance Blind TicTacToe.
Partially Observable Markov Decision Process (POMDP) is a framework applicable to many real world problems. In this work, we propose an approach to solve POMDPs with multimodal belief by relying on a policy that solves the fully observable version. By defininig a new, mixture value function based on the value function from the fully observable variant, we can use the corresponding greedy policy to solve the POMDP itself. We develop the mathematical framework necessary for discussion, and introduce a benchmark built on the task of Reconnaissance Blind TicTacToe. On this benchmark, we show that our policy outperforms policies ignoring the existence of multiple modes.