QMDP-Net: Deep Learning for Planning under Partial Observability
This addresses planning challenges in robotics and AI for partially observable environments, with incremental improvements in generalization and transfer.
The paper tackles planning under partial observability by introducing QMDP-net, a neural network architecture that combines model-free learning and model-based planning, showing strong performance in robotic simulation tasks and sometimes outperforming the QMDP algorithm.
This paper introduces the QMDP-net, a neural network architecture for planning under partial observability. The QMDP-net combines the strengths of model-free learning and model-based planning. It is a recurrent policy network, but it represents a policy for a parameterized set of tasks by connecting a model with a planning algorithm that solves the model, thus embedding the solution structure of planning in a network learning architecture. The QMDP-net is fully differentiable and allows for end-to-end training. We train a QMDP-net on different tasks so that it can generalize to new ones in the parameterized task set and "transfer" to other similar tasks beyond the set. In preliminary experiments, QMDP-net showed strong performance on several robotic tasks in simulation. Interestingly, while QMDP-net encodes the QMDP algorithm, it sometimes outperforms the QMDP algorithm in the experiments, as a result of end-to-end learning.