Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars
This work addresses the need for efficient and scalable virtual environments to train and test autonomous driving systems, though it is incremental in leveraging existing game engines.
The study tackled the problem of training and testing deep learning models for self-driving cars by generating over 480,000 labeled virtual images in Grand Theft Auto V, achieving encouraging results on 50,000 test images for tasks like following distance and lane detection. It also introduced the Princeton Virtual Environment (PVE) to simulate rare corner cases, such as the Florida Tesla crash, for enhancing model safety.
As an initial assessment, over 480,000 labeled virtual images of normal highway driving were readily generated in Grand Theft Auto V's virtual environment. Using these images, a CNN was trained to detect following distance to cars/objects ahead, lane markings, and driving angle (angular heading relative to lane centerline): all variables necessary for basic autonomous driving. Encouraging results were obtained when tested on over 50,000 labeled virtual images from substantially different GTA-V driving environments. This initial assessment begins to define both the range and scope of the labeled images needed for training as well as the range and scope of labeled images needed for testing the definition of boundaries and limitations of trained networks. It is the efficacy and flexibility of a "GTA-V"-like virtual environment that is expected to provide an efficient well-defined foundation for the training and testing of Convolutional Neural Networks for safe driving. Additionally, described is the Princeton Virtual Environment (PVE) for the training, testing and enhancement of safe driving AI, which is being developed using the video-game engine Unity. PVE is being developed to recreate rare but critical corner cases that can be used in re-training and enhancing machine learning models and understanding the limitations of current self driving models. The Florida Tesla crash is being used as an initial reference.