Deep Reinforcement Learning with Model Learning and Monte Carlo Tree Search in Minecraft
This work addresses sample efficiency in reinforcement learning for Minecraft tasks, but it is incremental as it adapts existing methods to a specific domain.
The authors tackled the problem of visual-input tasks in Minecraft by proposing a model-based approach that combines a learned DNN transition model with Monte Carlo tree search, achieving performance comparable to Deep Q-Network while learning faster and being more training sample efficient.
Deep reinforcement learning has been successfully applied to several visual-input tasks using model-free methods. In this paper, we propose a model-based approach that combines learning a DNN-based transition model with Monte Carlo tree search to solve a block-placing task in Minecraft. Our learned transition model predicts the next frame and the rewards one step ahead given the last four frames of the agent's first-person-view image and the current action. Then a Monte Carlo tree search algorithm uses this model to plan the best sequence of actions for the agent to perform. On the proposed task in Minecraft, our model-based approach reaches the performance comparable to the Deep Q-Network's, but learns faster and, thus, is more training sample efficient.