Monte Carlo Planning in Hybrid Belief POMDPs
This addresses a gap in online POMDP solvers for hybrid beliefs, which is important for robotics and AI applications dealing with uncertainty, though it appears incremental as it builds on existing MCTS methods.
The paper tackles the problem of planning in hybrid belief POMDPs, which involve both discrete and continuous variables and are computationally challenging due to exponential hypothesis growth, by introducing the HB-MCP algorithm that uses MCTS and UCB exploration to manage hybrid beliefs, achieving evaluation in simulated aliased environments.
Real-world problems often require reasoning about hybrid beliefs, over both discrete and continuous random variables. Yet, such a setting has hardly been investigated in the context of planning. Moreover, existing online Partially Observable Markov Decision Processes (POMDPs) solvers do not support hybrid beliefs directly. In particular, these solvers do not address the added computational burden due to an increasing number of hypotheses with the planning horizon, which can grow exponentially. As part of this work, we present a novel algorithm, Hybrid Belief Monte Carlo Planning (HB-MCP) that utilizes the Monte Carlo Tree Search (MCTS) algorithm to solve a POMDP while maintaining a hybrid belief. We illustrate how the upper confidence bound (UCB) exploration bonus can be leveraged to guide the growth of hypotheses trees alongside the belief trees. We then evaluate our approach in highly aliased simulated environments where unresolved data association leads to multi-modal belief hypotheses.