DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs
This work addresses the problem of robust decision-making under uncertainty in continuous POMDPs for robotics and AI applications, representing a novel method for a known bottleneck.
The paper tackled the challenge of inferring multi-modal state distributions and making planning uncertainty-dependent in continuous POMDPs, resulting in the DualSMC model that effectively handles complex observations like images and achieves strong performance in three continuous domains.
A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real states and the other over the future optimal trajectories, and combine the merits of these two parts in a new model named the DualSMC network. In particular, we first introduce an adversarial particle filter that leverages the adversarial relationship between its internal components. Based on the filtering results, we then propose a planning algorithm that extends the previous SMC planning approach [Piche et al., 2018] to continuous POMDPs with an uncertainty-dependent policy. Crucially, not only can DualSMC handle complex observations such as image input but also it remains highly interpretable. It is shown to be effective in three continuous POMDP domains: the floor positioning domain, the 3D light-dark navigation domain, and a modified Reacher domain.