A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithms
This work addresses multi-agent cooperative decision-making in complex traffic scenarios, offering an incremental improvement for autonomous vehicle systems.
The paper tackled the challenge of low sample efficiency in reinforcement learning for multi-vehicle cooperative driving by proposing a differentiated reward method based on steady-state transition systems, which accelerated training convergence and improved traffic efficiency, safety, and action rationality in algorithms like MAPPO, MADQN, and QMIX.
Reinforcement learning (RL) shows great potential for optimizing multi-vehicle cooperative driving strategies through the state-action-reward feedback loop, but it still faces challenges such as low sample efficiency. This paper proposes a differentiated reward method based on steady-state transition systems, which incorporates state transition gradient information into the reward design by analyzing traffic flow characteristics, aiming to optimize action selection and policy learning in multi-vehicle cooperative decision-making. The performance of the proposed method is validated in RL algorithms such as MAPPO, MADQN, and QMIX under varying autonomous vehicle penetration. The results show that the differentiated reward method significantly accelerates training convergence and outperforms centering reward and others in terms of traffic efficiency, safety, and action rationality. Additionally, the method demonstrates strong scalability and environmental adaptability, providing a novel approach for multi-agent cooperative decision-making in complex traffic scenarios.