Heuristic Search Value Iteration for POMDPs
This work addresses the computational bottleneck in POMDP planning for robotics and AI applications, offering a significant performance improvement.
The paper tackles the problem of planning in partially observable Markov decision processes (POMDPs) by introducing heuristic search value iteration (HSVI), an anytime algorithm that provides a policy with a provable regret bound. It demonstrates speedups of over 100 times compared to other state-of-the-art algorithms on benchmarks and scales to a rover exploration problem 10 times larger than typical POMDP problems.
We present a novel POMDP planning algorithm called heuristic search value iteration (HSVI).HSVI is an anytime algorithm that returns a policy and a provable bound on its regret with respect to the optimal policy. HSVI gets its power by combining two well-known techniques: attention-focusing search heuristics and piecewise linear convex representations of the value function. HSVI's soundness and convergence have been proven. On some benchmark problems from the literature, HSVI displays speedups of greater than 100 with respect to other state-of-the-art POMDP value iteration algorithms. We also apply HSVI to a new rover exploration problem 10 times larger than most POMDP problems in the literature.