Reinforcement Learning with Latent State Inference for Autonomous On-ramp Merging under Observation Delay
This addresses the challenge of safe and efficient merging for autonomous vehicles in dynamic traffic, with incremental improvements for handling observation delays.
This paper tackles the problem of autonomous on-ramp merging for self-driving vehicles by introducing the L3IS agent, which uses latent state inference to model unobservable aspects like driver intents, achieving a 99.90% success rate in simulations based on real highway data, and an augmented version, AL3IS, maintains a 93.84% success rate under 1-second communication delays.
This paper presents a novel approach to address the challenging problem of autonomous on-ramp merging, where a self-driving vehicle needs to seamlessly integrate into a flow of vehicles on a multi-lane highway. We introduce the Lane-keeping, Lane-changing with Latent-state Inference and Safety Controller (L3IS) agent, designed to perform the on-ramp merging task safely without comprehensive knowledge about surrounding vehicles' intents or driving styles. We also present an augmentation of this agent called AL3IS that accounts for observation delays, allowing the agent to make more robust decisions in real-world environments with vehicle-to-vehicle (V2V) communication delays. By modeling the unobservable aspects of the environment through latent states, such as other drivers' intents, our approach enhances the agent's ability to adapt to dynamic traffic conditions, optimize merging maneuvers, and ensure safe interactions with other vehicles. We demonstrate the effectiveness of our method through extensive simulations generated from real traffic data and compare its performance with existing approaches. L3IS shows a 99.90% success rate in a challenging on-ramp merging case generated from the real US Highway 101 data. We further perform a sensitivity analysis on AL3IS to evaluate its robustness against varying observation delays, which demonstrates an acceptable performance of 93.84% success rate in 1-second V2V communication delay.