Control of Memory, Active Perception, and Action in Minecraft
This work addresses challenges in reinforcement learning for AI agents in complex, partially observable worlds, but it is incremental as it builds on existing deep reinforcement learning architectures.
The paper tackles the problem of reinforcement learning in complex 3D environments like Minecraft, which involve challenges such as partial observability and delayed rewards, by introducing new tasks and memory-based architectures; the result shows that these architectures generalize better to unseen environments than existing methods.
In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world). We then use these tasks to systematically compare and contrast existing deep reinforcement learning (DRL) architectures with our new memory-based DRL architectures. These tasks are designed to emphasize, in a controllable manner, issues that pose challenges for RL methods including partial observability (due to first-person visual observations), delayed rewards, high-dimensional visual observations, and the need to use active perception in a correct manner so as to perform well in the tasks. While these tasks are conceptually simple to describe, by virtue of having all of these challenges simultaneously they are difficult for current DRL architectures. Additionally, we evaluate the generalization performance of the architectures on environments not used during training. The experimental results show that our new architectures generalize to unseen environments better than existing DRL architectures.