Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
This work addresses model-based reinforcement learning for scenarios where interaction costs are high, but it appears incremental as it builds on existing single-step transition models.
The paper tackles the problem of expensive environmental interaction in reinforcement learning by proposing a multi-step model that predicts outcomes of variable-length action sequences, showing preliminary results with a clear advantage over one-step models.
When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn a single-step transition model. In this paper, we propose a multi-step model that predicts the outcome of an action sequence with variable length. We show that this model is easy to learn, and that the model can make policy-conditional predictions. We report preliminary results that show a clear advantage for the multi-step model compared to its one-step counterpart.