A Finite-State Controller Based Offline Solver for Deterministic POMDPs
This addresses planning problems for agents with uncertain environmental states, such as in mobile robot forest mapping, but appears incremental as it adapts an existing algorithm to a specific domain.
The paper tackles the problem of solving deterministic partially observable Markov decision processes (DetPOMDPs) by proposing DetMCVI, an adaptation of Monte Carlo Value Iteration that builds finite-state controller policies, and reports that it solves large problems with a high success rate and outperforms existing baselines.
Deterministic partially observable Markov decision processes (DetPOMDPs) often arise in planning problems where the agent is uncertain about its environmental state but can act and observe deterministically. In this paper, we propose DetMCVI, an adaptation of the Monte Carlo Value Iteration (MCVI) algorithm for DetPOMDPs, which builds policies in the form of finite-state controllers (FSCs). DetMCVI solves large problems with a high success rate, outperforming existing baselines for DetPOMDPs. We also verify the performance of the algorithm in a real-world mobile robot forest mapping scenario.