SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement Learning
This addresses the challenge of applying model-based RL to complex domains with partial knowledge, offering a more flexible approach than existing hybrid methods.
The paper tackles the problem of using incomplete models in model-based reinforcement learning by proposing SAGE, an algorithm that combines symbolic planning with neural learning, achieving superior performance on Taxi World and Minecraft variations.
Model-based reinforcement learning algorithms are typically more sample efficient than their model-free counterparts, especially in sparse reward problems. Unfortunately, many interesting domains are too complex to specify the complete models required by traditional model-based approaches. Learning a model takes a large number of environment samples, and may not capture critical information if the environment is hard to explore. If we could specify an incomplete model and allow the agent to learn how best to use it, we could take advantage of our partial understanding of many domains. Existing hybrid planning and learning systems which address this problem often impose highly restrictive assumptions on the sorts of models which can be used, limiting their applicability to a wide range of domains. In this work we propose SAGE, an algorithm combining learning and planning to exploit a previously unusable class of incomplete models. This combines the strengths of symbolic planning and neural learning approaches in a novel way that outperforms competing methods on variations of taxi world and Minecraft.