Procedural generation of meta-reinforcement learning tasks
This work addresses the need for diverse and scalable environments in meta-learning research, particularly for reinforcement learning, though it is incremental as it builds on existing task types.
The authors tackled the limited availability of meta-learning environments by developing a parametrized space for generating simple meta-reinforcement learning tasks, enabling the creation of an infinite variety of novel tasks that include well-known benchmarks like bandit problems and T-mazes.
Open-endedness stands to benefit from the ability to generate an infinite variety of diverse, challenging environments. One particularly interesting type of challenge is meta-learning ("learning-to-learn"), a hallmark of intelligent behavior. However, the number of meta-learning environments in the literature is limited. Here we describe a parametrized space for simple meta-reinforcement learning (meta-RL) tasks with arbitrary stimuli. The parametrization allows us to randomly generate an arbitrary number of novel simple meta-learning tasks. The parametrization is expressive enough to include many well-known meta-RL tasks, such as bandit problems, the Harlow task, T-mazes, the Daw two-step task and others. Simple extensions allow it to capture tasks based on two-dimensional topological spaces, such as full mazes or find-the-spot domains. We describe a number of randomly generated meta-RL domains of varying complexity and discuss potential issues arising from random generation.