Constructing Abstraction Hierarchies Using a Skill-Symbol Loop
This work addresses the challenge of efficient planning in reinforcement learning by constructing abstraction hierarchies, but it appears incremental as it builds on existing results linking skills to abstract representations.
The authors tackled the problem of building abstraction hierarchies for planning by proposing a framework where an agent alternates between skill- and representation-acquisition phases to create increasingly abstract Markov decision processes, and they demonstrated its application in the Taxi domain for fast planning.
We describe a framework for building abstraction hierarchies whereby an agent alternates skill- and representation-acquisition phases to construct a sequence of increasingly abstract Markov decision processes. Our formulation builds on recent results showing that the appropriate abstract representation of a problem is specified by the agent's skills. We describe how such a hierarchy can be used for fast planning, and illustrate the construction of an appropriate hierarchy for the Taxi domain.