Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search
This addresses the challenge of enabling automated vehicles to cooperate implicitly in traffic, which is incremental as it builds on existing MCTS methods with macro-actions.
The paper tackles the problem of automated vehicles lacking implicit cooperation by proposing a decentralized cooperative planning approach using hierarchical Monte Carlo Tree Search (MCTS) with macro-actions, achieving effective cooperative planning in heterogeneous environments without predefined policies.
Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps and increase the effective search depth requiring fewer iterations to plan over longer horizons. Without predefined policies for macro-actions, the algorithm simultaneously learns policies over and within macro-actions. The proposed method is evaluated under several conflict scenarios, showing that the algorithm can achieve effective cooperative planning with learned macro-actions in heterogeneous environments.