Recurrent Environment Simulators
This improves environment simulators for agents in planning and exploration tasks, though it appears incremental as it builds on previous simulators.
The paper tackles the problem of simulating environment dynamics from high-dimensional pixel observations by introducing recurrent neural networks that make coherent long-term predictions, demonstrating adaptability across 10 Atari games, a 3D car racing environment, and complex 3D mazes.
Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions for hundreds of time-steps into the future. We present an in-depth analysis of the factors affecting performance, providing the most extensive attempt to advance the understanding of the properties of these models. We address the issue of computationally inefficiency with a model that does not need to generate a high-dimensional image at each time-step. We show that our approach can be used to improve exploration and is adaptable to many diverse environments, namely 10 Atari games, a 3D car racing environment, and complex 3D mazes.