Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learning
This work addresses the problem of generalizability in automated driving systems for complex traffic environments, though it appears incremental as it builds on existing DRL methods with a hierarchical approach.
The paper tackles the challenge of automated driving in complex traffic scenarios by introducing a hierarchical reinforcement learning framework that decomposes decision-making into high-level and low-level controllers, demonstrating its superiority in managing complex highway driving through simulation experiments.
Developing an automated driving system capable of navigating complex traffic environments remains a formidable challenge. Unlike rule-based or supervised learning-based methods, Deep Reinforcement Learning (DRL) based controllers eliminate the need for domain-specific knowledge and datasets, thus providing adaptability to various scenarios. Nonetheless, a common limitation of existing studies on DRL-based controllers is their focus on driving scenarios with simple traffic patterns, which hinders their capability to effectively handle complex driving environments with delayed, long-term rewards, thus compromising the generalizability of their findings. In response to these limitations, our research introduces a pioneering hierarchical framework that efficiently decomposes intricate decision-making problems into manageable and interpretable subtasks. We adopt a two step training process that trains the high-level controller and low-level controller separately. The high-level controller exhibits an enhanced exploration potential with long-term delayed rewards, and the low-level controller provides longitudinal and lateral control ability using short-term instantaneous rewards. Through simulation experiments, we demonstrate the superiority of our hierarchical controller in managing complex highway driving situations.