Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion
For biped robot locomotion, this work shows that incorporating passive body dynamics can improve robustness and energy efficiency, but the approach is incremental as it applies existing model-based RL to a specific robot design.
This study uses model-based deep reinforcement learning to generate walking and running in biped robots with passive elements (springs). The passive elements enable robust, energy-efficient locomotion by exploiting stable limit cycles from body-ground interaction, though training converges slowly due to attractor dynamics.
Embodiment is a significant keyword in recent machine learning fields. This study focused on the passive nature of the body of a biped robot to generate walking and running locomotion using model-based deep reinforcement learning. We constructed two models in a simulator, one with passive elements (e.g., springs) and the other, which is similar to general humanoids, without passive elements. The training of the model with passive elements was highly affected by the attractor of the system. This lead that although the trajectories quickly converged to limit cycles, it took a long time to obtain large rewards. However, thanks to the attractor-driven learning, the acquired locomotion was robust and energy-efficient. The results revealed that robots with passive elements could efficiently acquire high-performance locomotion by utilizing stable limit cycles generated through dynamic interaction between the body and ground. This study demonstrates the importance of implementing passive properties in the body for future embodied AI.