Identifying Important Sensory Feedback for Learning Locomotion Skills
This work provides quantitative guidelines for robot motor learning, addressing a lack of analysis in state selection for locomotion skills, though it is incremental as it builds on existing DRL methods.
The authors tackled the problem of identifying essential sensory feedback for learning robot locomotion skills through deep reinforcement learning, and demonstrated that using only key states like joint positions and velocities achieves robust performance comparable to using all states, with significant drops if key states are missing.
Robot motor skills can be learned through deep reinforcement learning (DRL) by neural networks as state-action mappings. While the selection of state observations is crucial, there has been a lack of quantitative analysis to date. Here, we present a systematic saliency analysis that quantitatively evaluates the relative importance of different feedback states for motor skills learned through DRL. Our approach can identify the most essential feedback states for locomotion skills, including balance recovery, trotting, bounding, pacing and galloping. By using only key states including joint positions, gravity vector, base linear and angular velocities, we demonstrate that a simulated quadruped robot can achieve robust performance in various test scenarios across these distinct skills. The benchmarks using task performance metrics show that locomotion skills learned with key states can achieve comparable performance to those with all states, and the task performance or learning success rate will drop significantly if key states are missing. This work provides quantitative insights into the relationship between state observations and specific types of motor skills, serving as a guideline for robot motor learning. The proposed method is applicable to differentiable state-action mapping, such as neural network based control policies, enabling the learning of a wide range of motor skills with minimal sensing dependencies.