AIApr 13, 2022
Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agentsOlivier Moulin, Vincent Francois-Lavet, Paul Elbers et al.
Adapting a Reinforcement Learning (RL) agent to an unseen environment is a difficult task due to typical over-fitting on the training environment. RL agents are often capable of solving environments very close to the trained environment, but when environments become substantially different, their performance quickly drops. When agents are retrained on new environments, a second issue arises: there is a risk of catastrophic forgetting, where the performance on previously seen environments is seriously hampered. This paper proposes a novel approach that exploits an eco-system of agents to address both concerns. Hereby, the (limited) adaptive power of individual agents is harvested to build a highly adaptive eco-system.
AIDec 13, 2022
Improving generalization in reinforcement learning through forked agentsOlivier Moulin, Vincent Francois-Lavet, Mark Hoogendoorn
An eco-system of agents each having their own policy with some, but limited, generalizability has proven to be a reliable approach to increase generalization across procedurally generated environments. In such an approach, new agents are regularly added to the eco-system when encountering a new environment that is outside of the scope of the eco-system. The speed of adaptation and general effectiveness of the eco-system approach highly depends on the initialization of new agents. In this paper we propose different initialization techniques, inspired from Deep Neural Network initialization and transfer learning, and study their impact.
LGNov 25, 2025
Leveraging weights signals -- Predicting and improving generalizability in reinforcement learningOlivier Moulin, Vincent Francois-lavet, Paul Elbers et al.
Generalizability of Reinforcement Learning (RL) agents (ability to perform on environments different from the ones they have been trained on) is a key problem as agents have the tendency to overfit to their training environments. In order to address this problem and offer a solution to increase the generalizability of RL agents, we introduce a new methodology to predict the generalizability score of RL agents based on the internal weights of the agent's neural networks. Using this prediction capability, we propose some changes in the Proximal Policy Optimization (PPO) loss function to boost the generalization score of the agents trained with this upgraded version. Experimental results demonstrate that our improved PPO algorithm yields agents with stronger generalizability compared to the original version.