An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep Reinforcement Learning
This work addresses energy efficiency for snake robots used in industrial applications like life detection in narrow spaces, but it is incremental as it applies existing DRL methods to a specific robot control task.
The paper tackled the problem of energy-inefficient control in snake robots navigating unknown environments by developing a deep reinforcement learning gait policy, resulting in faster movement and more energy-efficient locomotion compared to conventional strategies.
Snake robots, comprised of sequentially connected joint actuators, have recently gained increasing attention in the industrial field, like life detection in narrow space. Such robots can navigate through the complex environment via the cooperation of multiple motors located on the backbone. However, controlling the robots in an unknown environment is challenging, and conventional control strategies can be energy inefficient or even fail to navigate to the destination. In this work, a snake locomotion gait policy is developed via deep reinforcement learning (DRL) for energy-efficient control. We apply proximal policy optimization (PPO) to each joint motor parameterized by angular velocity and the DRL agent learns the standard serpenoid curve at each timestep. The robot simulator and task environment are built upon PyBullet. Comparing to conventional control strategies, the snake robots controlled by the trained PPO agent can achieve faster movement and more energy-efficient locomotion gait. This work demonstrates that DRL provides an energy-efficient solution for robot control.