Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spaces
This work addresses motion planning under uncertainty for automated driving, representing an incremental improvement by adapting existing bandit methods to continuous action spaces.
The paper tackles the computational intractability of solving partially observable Markov decision processes (POMDPs) by introducing Lipschitz bandit heuristics that exploit correlations between similar actions in continuous spaces, demonstrating effectiveness in automated driving motion planning.
Decision making under uncertainty can be framed as a partially observable Markov decision process (POMDP). Finding exact solutions of POMDPs is generally computationally intractable, but the solution can be approximated by sampling-based approaches. These sampling-based POMDP solvers rely on multi-armed bandit (MAB) heuristics, which assume the outcomes of different actions to be uncorrelated. In some applications, like motion planning in continuous spaces, similar actions yield similar outcomes. In this paper, we utilize variants of MAB heuristics that make Lipschitz continuity assumptions on the outcomes of actions to improve the efficiency of sampling-based planning approaches. We demonstrate the effectiveness of this approach in the context of motion planning for automated driving.