From model-based learning to model-free behaviour with Meta-Interpretive Learning
This work addresses the problem of enabling autonomous agents to act in novel environments by integrating model-based and model-free approaches, though it appears incremental as it builds on existing learning methods.
The paper tackles the challenge of creating autonomous agents that combine model-based planning and model-free action capabilities by using Meta-Interpretive Learning to train a model-free Controller from a model-based Solver, demonstrating equivalence in solving grid navigation problems such as mazes and lake maps.
A "model" is a theory that describes the state of an environment and the effects of an agent's decisions on the environment. A model-based agent can use its model to predict the effects of its future actions and so plan ahead, but must know the state of the environment. A model-free agent cannot plan, but can act without a model and without completely observing the environment. An autonomous agent capable of acting independently in novel environments must combine both sets of capabilities. We show how to create such an agent with Meta-Interpretive Learning used to learn a model-based Solver used to train a model-free Controller that can solve the same planning problems as the Solver. We demonstrate the equivalence in problem-solving ability of the two agents on grid navigation problems in two kinds of environment: randomly generated mazes, and lake maps with wide open areas. We find that all navigation problems solved by the Solver are also solved by the Controller, indicating the two are equivalent.