Deep Learning of Robotic Tasks without a Simulator using Strong and Weak Human Supervision
This work addresses the challenge of enabling robots to learn complex tasks like driving from human guidance, though it is incremental as it builds on existing imitation and reinforcement learning methods.
The authors tackled the problem of training a robotic agent for autonomous highway steering without a simulator by combining human supervision and deep learning, resulting in a successful implementation in the game Assetto Corsa that operates in real-time using only vision.
We propose a scheme for training a computerized agent to perform complex human tasks such as highway steering. The scheme is designed to follow a natural learning process whereby a human instructor teaches a computerized trainee. The learning process consists of five elements: (i) unsupervised feature learning; (ii) supervised imitation learning; (iii) supervised reward induction; (iv) supervised safety module construction; and (v) reinforcement learning. We implemented the last four elements of the scheme using deep convolutional networks and applied it to successfully create a computerized agent capable of autonomous highway steering over the well-known racing game Assetto Corsa. We demonstrate that the use of the last four elements is essential to effectively carry out the steering task using vision alone, without access to a driving simulator internals, and operating in wall-clock time. This is made possible also through the introduction of a safety network, a novel way for preventing the agent from performing catastrophic mistakes during the reinforcement learning stage.