The Dreaming Variational Autoencoder for Reinforcement Learning Environments
This addresses exploration and planning problems in reinforcement learning for researchers, but it appears incremental as it builds on existing generative methods and benchmarks.
The paper tackles challenges in reinforcement learning, such as exploration and planning, by introducing The Dreaming Variational Autoencoder (DVAE) for generative modeling in sparse-feedback environments and Deep Maze as a novel benchmark, showing initial findings to encourage further work.
Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.