Reciprocal Collision Avoidance for General Nonlinear Agents using Reinforcement Learning
This work addresses collision avoidance in decentralized settings for general nonlinear agents, offering a practical solution for robotics and autonomous systems, though it is incremental by building on existing RL and ORCA methods.
The paper tackles decentralized collision avoidance for multiple nonlinear agents with continuous action spaces by combining reinforcement learning for two-agent scenarios with optimal reciprocal collision avoidance constraints for multi-agent extension, achieving competitive performance in simulations with smooth trajectories and reduced congestion and deadlock.
Finding feasible and collision-free paths for multiple nonlinear agents is challenging in the decentralized scenarios due to limited available information of other agents and complex dynamics constraints. In this paper, we propose a fast multi-agent collision avoidance algorithm for general nonlinear agents with continuous action space, where each agent observes only positions and velocities of nearby agents. To reduce online computation, we first decompose the multi-agent scenario and solve a two agents collision avoidance problem using reinforcement learning (RL). When extending the trained policy to a multi-agent problem, safety is ensured by introducing the optimal reciprocal collision avoidance (ORCA) as linear constraints and the overall collision avoidance action could be found through simple convex optimization. Most existing RL-based multi-agent collision avoidance algorithms rely on the direct control of agent velocities. In sharp contrasts, our approach is applicable to general nonlinear agents. Realistic simulations based on nonlinear bicycle agent models are performed with various challenging scenarios, indicating a competitive performance of the proposed method in avoiding collisions, congestion and deadlock with smooth trajectories.