Online algorithms for POMDPs with continuous state, action, and observation spaces
This work addresses the challenge of online planning in continuous-state POMDPs, which is important for robotics and control, but the proposed solution is incremental.
The paper identifies a collapse issue in double progressive widening for continuous-space POMDPs and proposes two new algorithms, POMCPOW and PFT-DPW, that use weighted particle filtering to overcome it. Simulations show these algorithms succeed where previous approaches fail.
Online solvers for partially observable Markov decision processes have been applied to problems with large discrete state spaces, but continuous state, action, and observation spaces remain a challenge. This paper begins by investigating double progressive widening (DPW) as a solution to this challenge. However, we prove that this modification alone is not sufficient because the belief representations in the search tree collapse to a single particle causing the algorithm to converge to a policy that is suboptimal regardless of the computation time. This paper proposes and evaluates two new algorithms, POMCPOW and PFT-DPW, that overcome this deficiency by using weighted particle filtering. Simulation results show that these modifications allow the algorithms to be successful where previous approaches fail.