Improved POMDP Tree Search Planning with Prioritized Action Branching
This addresses a scalability issue for researchers and practitioners in robotics and AI planning, but it is incremental as it builds on prior tree search methods.
The paper tackles the challenge of scaling online POMDP solvers to large action spaces by proposing PA-POMCPOW, which samples actions based on a score combining expected reward and information gain, resulting in outperforming existing state-of-the-art solvers on such problems.
Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. This paper proposes a method called PA-POMCPOW to sample a subset of the action space that provides varying mixtures of exploitation and exploration for inclusion in a search tree. The proposed method first evaluates the action space according to a score function that is a linear combination of expected reward and expected information gain. The actions with the highest score are then added to the search tree during tree expansion. Experiments show that PA-POMCPOW is able to outperform existing state-of-the-art solvers on problems with large discrete action spaces.