ES-Parkour: Advanced Robot Parkour with Bio-inspired Event Camera and Spiking Neural Network
This work addresses energy efficiency and robustness in robotic perception and control for demanding environments like outdoor parkour, representing a novel integration rather than a foundational breakthrough.
The paper tackled the problem of quadruped robot parkour by integrating event cameras and spiking neural networks to overcome limitations of traditional visual sensors and deep neural networks, resulting in a model that achieves excellent parkour performance with only 11.7% of the energy consumption of an ANN-based model.
In recent years, quadruped robotics has advanced significantly, particularly in perception and motion control via reinforcement learning, enabling complex motions in challenging environments. Visual sensors like depth cameras enhance stability and robustness but face limitations, such as low operating frequencies relative to joint control and sensitivity to lighting, which hinder outdoor deployment. Additionally, deep neural networks in sensor and control systems increase computational demands. To address these issues, we introduce spiking neural networks (SNNs) and event cameras to perform a challenging quadruped parkour task. Event cameras capture dynamic visual data, while SNNs efficiently process spike sequences, mimicking biological perception. Experimental results demonstrate that this approach significantly outperforms traditional models, achieving excellent parkour performance with just 11.7% of the energy consumption of an artificial neural network (ANN)-based model, yielding an 88.3% energy reduction. By integrating event cameras with SNNs, our work advances robotic reinforcement learning and opens new possibilities for applications in demanding environments.