Adaptable Agent Populations via a Generative Model of Policies
This work addresses the need for adaptable and robust agent populations in AI, offering a method to generate diverse policies without separate parameters, which is incremental as it builds on generative modeling for policy spaces.
The paper tackles the problem of learning diverse and high-reward policies in environments by introducing a generative model that maps a latent space to agent policies, enabling adaptation to environmental changes through latent space selection. It demonstrates capabilities in environments like an open-ended grid-world and a two-player soccer environment, with code and experiments provided.
In the natural world, life has found innumerable ways to survive and often thrive. Between and even within species, each individual is in some manner unique, and this diversity lends adaptability and robustness to life. In this work, we aim to learn a space of diverse and high-reward policies on any given environment. To this end, we introduce a generative model of policies, which maps a low-dimensional latent space to an agent policy space. Our method enables learning an entire population of agent policies, without requiring the use of separate policy parameters. Just as real world populations can adapt and evolve via natural selection, our method is able to adapt to changes in our environment solely by selecting for policies in latent space. We test our generative model's capabilities in a variety of environments, including an open-ended grid-world and a two-player soccer environment. Code, visualizations, and additional experiments can be found at https://kennyderek.github.io/adap/.