Anytime Incremental $ρ$POMDP Planning in Continuous Spaces
This addresses a critical bottleneck in decision-making under uncertainty for applications like autonomous driving and robotic exploration, though it is incremental as it builds on existing $\rho$POMDP frameworks.
The paper tackles the problem of limited adaptability in online $\rho$POMDP solvers for continuous spaces by introducing $\rho$POMCPOW, an anytime solver that dynamically refines belief representations with formal guarantees, and it outperforms state-of-the-art solvers in efficiency and solution quality.
Partially Observable Markov Decision Processes (POMDPs) provide a robust framework for decision-making under uncertainty in applications such as autonomous driving and robotic exploration. Their extension, $ρ$POMDPs, introduces belief-dependent rewards, enabling explicit reasoning about uncertainty. Existing online $ρ$POMDP solvers for continuous spaces rely on fixed belief representations, limiting adaptability and refinement - critical for tasks such as information-gathering. We present $ρ$POMCPOW, an anytime solver that dynamically refines belief representations, with formal guarantees of improvement over time. To mitigate the high computational cost of updating belief-dependent rewards, we propose a novel incremental computation approach. We demonstrate its effectiveness for common entropy estimators, reducing computational cost by orders of magnitude. Experimental results show that $ρ$POMCPOW outperforms state-of-the-art solvers in both efficiency and solution quality.