Will it Blend? Composing Value Functions in Reinforcement Learning
This addresses the challenge of lifelong learning for AI agents by enabling principled skill composition, though it appears incremental as it builds on existing RL frameworks.
The paper tackles the problem of composing existing skills to solve new tasks in reinforcement learning by providing a recipe for optimal value function composition, first in entropy-regularized RL and then extending it to standard RL, and demonstrates that an agent can solve new tasks without further learning in a video game environment.
An important property for lifelong-learning agents is the ability to combine existing skills to solve unseen tasks. In general, however, it is unclear how to compose skills in a principled way. We provide a "recipe" for optimal value function composition in entropy-regularised reinforcement learning (RL) and then extend this to the standard RL setting. Composition is demonstrated in a video game environment, where an agent with an existing library of policies is able to solve new tasks without the need for further learning.