Modular Recurrence in Contextual MDPs for Universal Morphology Control
This work addresses the problem of multi-robot control for researchers and practitioners by enabling better generalization to new robots, though it appears incremental as it builds on existing modular and recurrent approaches.
The paper tackled the challenge of generalizing a universal controller to unseen robot morphologies by hypothesizing that contextual information is partially observable and can be inferred through interactions, resulting in substantial improved performance on robots with unseen dynamics, kinematics, and topologies across four environments.
A universal controller for any robot morphology would greatly improve computational and data efficiency. By utilizing contextual information about the properties of individual robots and exploiting their modular structure in the architecture of deep reinforcement learning agents, steps have been made towards multi-robot control. Generalization to new, unseen robots, however, remains a challenge. In this paper we hypothesize that the relevant contextual information is partially observable, but that it can be inferred through interactions for better generalization to contexts that are not seen during training. To this extent, we implement a modular recurrent architecture and evaluate its generalization performance on a large set of MuJoCo robots. The results show a substantial improved performance on robots with unseen dynamics, kinematics, and topologies, in four different environments.