Scaling Internal-State Policy-Gradient Methods for POMDPs
This work addresses a key bottleneck in reinforcement learning for real-world, partially observable environments, though it appears incremental as it builds on existing policy-gradient methods.
The paper tackles the challenge of scaling policy-gradient methods for partially observable Markov decision processes (POMDPs) when memory is required, developing improved algorithms that show promise in large-scale applications like robot navigation and multi-agent problems.
Policy-gradient methods have received increased attention recently as a mechanism for learning to act in partially observable environments. They have shown promise for problems admitting memoryless policies but have been less successful when memory is required. In this paper we develop several improved algorithms for learning policies with memory in an infinite-horizon setting -- directly when a known model of the environment is available, and via simulation otherwise. We compare these algorithms on some large POMDPs, including noisy robot navigation and multi-agent problems.