Reinforcement learning for freeform robot design
This enables more flexible robot design for robotics researchers, though it's incremental as it builds on prior morphological adaptation work and currently only demonstrates open-loop control.
The paper tackles the problem of using reinforcement learning to optimize 3D robot morphology beyond simple limb adjustments, achieving freeform design of arbitrary external and internal structures through policy gradients that manipulate atomic building blocks.
Inspired by the necessity of morphological adaptation in animals, a growing body of work has attempted to expand robot training to encompass physical aspects of a robot's design. However, reinforcement learning methods capable of optimizing the 3D morphology of a robot have been restricted to reorienting or resizing the limbs of a predetermined and static topological genus. Here we show policy gradients for designing freeform robots with arbitrary external and internal structure. This is achieved through actions that deposit or remove bundles of atomic building blocks to form higher-level nonparametric macrostructures such as appendages, organs and cavities. Although results are provided for open loop control only, we discuss how this method could be adapted for closed loop control and sim2real transfer to physical machines in future.