Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds
This work addresses the problem of safe and efficient robot navigation in crowded environments for robotics applications, representing an incremental improvement by combining existing methods.
The paper tackles robot navigation in dense crowds by developing a Deep Reinforcement Learning policy that integrates dynamic feasibility and spatial awareness, resulting in a 33% increase in success rate and a 61% decrease in dynamics constraint violations.
We present a novel Deep Reinforcement Learning (DRL) based policy to compute dynamically feasible and spatially aware velocities for a robot navigating among mobile obstacles. Our approach combines the benefits of the Dynamic Window Approach (DWA) in terms of satisfying the robot's dynamics constraints with state-of-the-art DRL-based navigation methods that can handle moving obstacles and pedestrians well. Our formulation achieves these goals by embedding the environmental obstacles' motions in a novel low-dimensional observation space. It also uses a novel reward function to positively reinforce velocities that move the robot away from the obstacle's heading direction leading to significantly lower number of collisions. We evaluate our method in realistic 3-D simulated environments and on a real differential drive robot in challenging dense indoor scenarios with several walking pedestrians. We compare our method with state-of-the-art collision avoidance methods and observe significant improvements in terms of success rate (up to 33\% increase), number of dynamics constraint violations (up to 61\% decrease), and smoothness. We also conduct ablation studies to highlight the advantages of our observation space formulation, and reward structure.