Dynamic Obstacle Avoidance with Bounded Rationality Adversarial Reinforcement Learning
This work addresses the problem of robust obstacle avoidance for quadruped robots in dynamic environments, representing an incremental improvement over existing hierarchical RL approaches.
The authors tackled robust navigation for legged robots in unseen environments with dynamic obstacles by proposing Hi-QARL, a hierarchical adversarial reinforcement learning method that models obstacles as bounded-rationality adversaries, and demonstrated its effectiveness in randomized mazes and simulation on a Unitree GO1 robot.
Reinforcement Learning (RL) has proven largely effective in obtaining stable locomotion gaits for legged robots. However, designing control algorithms which can robustly navigate unseen environments with obstacles remains an ongoing problem within quadruped locomotion. To tackle this, it is convenient to solve navigation tasks by means of a hierarchical approach with a low-level locomotion policy and a high-level navigation policy. Crucially, the high-level policy needs to be robust to dynamic obstacles along the path of the agent. In this work, we propose a novel way to endow navigation policies with robustness by a training process that models obstacles as adversarial agents, following the adversarial RL paradigm. Importantly, to improve the reliability of the training process, we bound the rationality of the adversarial agent resorting to quantal response equilibria, and place a curriculum over its rationality. We called this method Hierarchical policies via Quantal response Adversarial Reinforcement Learning (Hi-QARL). We demonstrate the robustness of our method by benchmarking it in unseen randomized mazes with multiple obstacles. To prove its applicability in real scenarios, our method is applied on a Unitree GO1 robot in simulation.