Reinforcement Learning Generalization with Surprise Minimization
This addresses generalization issues in reinforcement learning for AI systems, but it is incremental as it builds on existing methods with a new reward component.
The paper tackled the problem of generalization gaps in deep reinforcement learning when agents face unseen or perturbed environments, and showed that a surprise minimizing agent with an additional reward from a simple density model achieves robustness in procedurally generated game environments.
Generalization remains a challenging problem for deep reinforcement learning algorithms, which are often trained and tested on the same set of deterministic game environments. When test environments are unseen and perturbed but the nature of the task remains the same, generalization gaps can arise. In this work, we propose and evaluate a surprise minimizing agent on a generalization benchmark to show an additional reward learned from a simple density model can show robustness in procedurally generated game environments that provide constant source of entropy and stochasticity.