Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation
This work addresses energy-efficient locomotion for hexapod robots, particularly in edge applications, though it appears incremental by combining existing SNN and CPG concepts.
The paper tackled the challenge of learning locomotion for legged robots under performance and energy constraints by proposing a reinforcement learning-based method to train a spiking central pattern generator (CPG) on a lightweight platform, achieving end-to-end learning without specifying concrete performance numbers.
Learning to walk -- i.e., learning locomotion under performance and energy constraints continues to be a challenge in legged robotics. Methods such as stochastic gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. These techniques are computationally intensive and often prohibitive for edge applications. These methods rely on complex sensors and pre-processing of data, which further increases energy and latency. Recent advances in spiking neural networks (SNNs) promise a significant reduction in computing owing to the sparse firing of neuros and has been shown to integrate reinforcement learning mechanisms with biologically observed spike time dependent plasticity (STDP). However, training a legged robot to walk by learning the synchronization patterns of central pattern generators (CPG) in an SNN framework has not been shown. This can marry the efficiency of SNNs with synchronized locomotion of CPG based systems providing breakthrough end-to-end learning in mobile robotics. In this paper, we propose a reinforcement based stochastic weight update technique for training a spiking CPG. The whole system is implemented on a lightweight raspberry pi platform with integrated sensors, thus opening up exciting new possibilities.