Zero-Shot Reinforcement Learning via Function Encoders
This addresses the problem of enabling RL agents to transfer knowledge across tasks without retraining, which is incremental as it builds on existing RL algorithms with a novel representation method.
The paper tackles the challenge of zero-shot transfer across related tasks in reinforcement learning by introducing a function encoder that represents tasks as weighted combinations of learned basis functions, enabling agents to achieve state-of-the-art data efficiency, asymptotic performance, and training stability without additional training.
Although reinforcement learning (RL) can solve many challenging sequential decision making problems, achieving zero-shot transfer across related tasks remains a challenge. The difficulty lies in finding a good representation for the current task so that the agent understands how it relates to previously seen tasks. To achieve zero-shot transfer, we introduce the function encoder, a representation learning algorithm which represents a function as a weighted combination of learned, non-linear basis functions. By using a function encoder to represent the reward function or the transition function, the agent has information on how the current task relates to previously seen tasks via a coherent vector representation. Thus, the agent is able to achieve transfer between related tasks at run time with no additional training. We demonstrate state-of-the-art data efficiency, asymptotic performance, and training stability in three RL fields by augmenting basic RL algorithms with a function encoder task representation.