Online algorithms for POMDPs with continuous state, action, and observation spaces
This work addresses a critical problem for robotics and AI applications requiring real-time decision-making under uncertainty, representing an incremental improvement over existing methods.
The paper tackled the challenge of online solvers for POMDPs with continuous state, action, and observation spaces by proposing new algorithms, POMCPOW and PFT-DPW, which use weighted particle filtering to overcome deficiencies in previous methods like double progressive widening, enabling success where prior approaches failed.
Online solvers for partially observable Markov decision processes have been applied to problems with large discrete state spaces, but continuous state, action, and observation spaces remain a challenge. This paper begins by investigating double progressive widening (DPW) as a solution to this challenge. However, we prove that this modification alone is not sufficient because the belief representations in the search tree collapse to a single particle causing the algorithm to converge to a policy that is suboptimal regardless of the computation time. This paper proposes and evaluates two new algorithms, POMCPOW and PFT-DPW, that overcome this deficiency by using weighted particle filtering. Simulation results show that these modifications allow the algorithms to be successful where previous approaches fail.