Generalization of Agent Behavior through Explicit Representation of Context
This addresses the generalization challenge for deploying autonomous agents in real-world tasks, offering a principled approach for continual adaptation, though it appears incremental as it builds on existing modular methods.
The paper tackles the problem of autonomous agents failing to extrapolate beyond their training data in interactive environments by co-evolving a context module with a skill module to recognize temporal variation and modulate actions. The result is significantly more robust behavior in unseen situations, demonstrated in Flappy Bird, LunarLander, and CARLA simulations.
In order to deploy autonomous agents in digital interactive environments, they must be able to act robustly in unseen situations. The standard machine learning approach is to include as much variation as possible into training these agents. The agents can then interpolate within their training, but they cannot extrapolate much beyond it. This paper proposes a principled approach where a context module is coevolved with a skill module in the game. The context module recognizes the temporal variation in the game and modulates the outputs of the skill module so that the action decisions can be made robustly even in previously unseen situations. The approach is evaluated in the Flappy Bird and LunarLander video games, as well as in the CARLA autonomous driving simulation. The Context+Skill approach leads to significantly more robust behavior in environments that require extrapolation beyond training. Such a principled generalization ability is essential in deploying autonomous agents in real-world tasks, and can serve as a foundation for continual adaptation as well.