Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations
This work addresses the critical problem of robustness in offline reinforcement learning for robot control, highlighting vulnerabilities that could impact real-world deployment.
The study evaluated the robustness of existing offline reinforcement learning methods for robot control against simulated joint actuator faults, finding they are significantly more vulnerable to action perturbations than online methods.
Offline reinforcement learning, which learns solely from datasets without environmental interaction, has gained attention. This approach, similar to traditional online deep reinforcement learning, is particularly promising for robot control applications. Nevertheless, its robustness against real-world challenges, such as joint actuator faults in robots, remains a critical concern. This study evaluates the robustness of existing offline reinforcement learning methods using legged robots from OpenAI Gym based on average episodic rewards. For robustness evaluation, we simulate failures by incorporating both random and adversarial perturbations, representing worst-case scenarios, into the joint torque signals. Our experiments show that existing offline reinforcement learning methods exhibit significant vulnerabilities to these action perturbations and are more vulnerable than online reinforcement learning methods, highlighting the need for more robust approaches in this field.