GRAM: Generalization in Deep RL with a Robust Adaptation Module
This addresses the challenge of reliable deployment of deep RL in varied conditions, though it appears incremental as it builds on existing generalization methods.
The paper tackles the problem of generalization in deep reinforcement learning for real-world deployment by proposing a framework that unifies in-distribution and out-of-distribution generalization, achieving strong performance in simulation and hardware experiments on a quadruped robot.
The reliable deployment of deep reinforcement learning in real-world settings requires the ability to generalize across a variety of conditions, including both in-distribution scenarios seen during training as well as novel out-of-distribution scenarios. In this work, we present a framework for dynamics generalization in deep reinforcement learning that unifies these two distinct types of generalization within a single architecture. We introduce a robust adaptation module that provides a mechanism for identifying and reacting to both in-distribution and out-of-distribution environment dynamics, along with a joint training pipeline that combines the goals of in-distribution adaptation and out-of-distribution robustness. Our algorithm GRAM achieves strong generalization performance across in-distribution and out-of-distribution scenarios upon deployment, which we demonstrate through extensive simulation and hardware locomotion experiments on a quadruped robot.