Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages
This addresses safety and interpretability issues in autonomous driving for deployment, but it is incremental as it builds on existing reinforcement learning methods with added supervision.
The paper tackles the problem of deploying neural network-based control in autonomous vehicles by introducing a reinforcement learning approach for longitudinal control that uses rule-based safety cages for weak supervision, resulting in improved safety, convergence speed, and model performance, with the safety cages enabling safe policy learning even with constrained or sub-optimal parameters.
The use of neural networks and reinforcement learning has become increasingly popular in autonomous vehicle control. However, the opaqueness of the resulting control policies presents a significant barrier to deploying neural network-based control in autonomous vehicles. In this paper, we present a reinforcement learning based approach to autonomous vehicle longitudinal control, where the rule-based safety cages provide enhanced safety for the vehicle as well as weak supervision to the reinforcement learning agent. By guiding the agent to meaningful states and actions, this weak supervision improves the convergence during training and enhances the safety of the final trained policy. This rule-based supervisory controller has the further advantage of being fully interpretable, thereby enabling traditional validation and verification approaches to ensure the safety of the vehicle. We compare models with and without safety cages, as well as models with optimal and constrained model parameters, and show that the weak supervision consistently improves the safety of exploration, speed of convergence, and model performance. Additionally, we show that when the model parameters are constrained or sub-optimal, the safety cages can enable a model to learn a safe driving policy even when the model could not be trained to drive through reinforcement learning alone.