Learning and Transfer of Modulated Locomotor Controllers
This work addresses the problem of efficient and robust locomotion control in robotics, offering a novel approach that improves learning in complex, high-dimensional simulated environments.
The paper tackles the challenge of learning locomotion tasks by proposing a two-level neural network architecture with a pre-trained 'spinal' module and a modulating 'cortical' network, which succeeds where monolithic architectures fail, enabling effective exploration and learning from sparse rewards across simulated bodies like a snake, quadruped, and humanoid.
We study a novel architecture and training procedure for locomotion tasks. A high-frequency, low-level "spinal" network with access to proprioceptive sensors learns sensorimotor primitives by training on simple tasks. This pre-trained module is fixed and connected to a low-frequency, high-level "cortical" network, with access to all sensors, which drives behavior by modulating the inputs to the spinal network. Where a monolithic end-to-end architecture fails completely, learning with a pre-trained spinal module succeeds at multiple high-level tasks, and enables the effective exploration required to learn from sparse rewards. We test our proposed architecture on three simulated bodies: a 16-dimensional swimming snake, a 20-dimensional quadruped, and a 54-dimensional humanoid. Our results are illustrated in the accompanying video at https://youtu.be/sboPYvhpraQ