SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning
This work addresses autonomous driving by improving policy learning through side task supervision, but it is incremental as it builds on existing representation learning methods.
The paper tackles learning visual driving policies conditioned on turning commands by using side tasks on semantics and object affordances to train a representation, and introduces a more realistic evaluation protocol in CARLA simulator that requires adherence to traffic rules for success.
We describe a policy learning approach to map visual inputs to driving controls conditioned on turning command that leverages side tasks on semantics and object affordances via a learned representation trained for driving. To learn this representation, we train a squeeze network to drive using annotations for the side task as input. This representation encodes the driving-relevant information associated with the side task while ideally throwing out side task-relevant but driving-irrelevant nuisances. We then train a mimic network to drive using only images as input and use the squeeze network's latent representation to supervise the mimic network via a mimicking loss. Notably, we do not aim to achieve the side task nor to learn features for it; instead, we aim to learn, via the mimicking loss, a representation of the side task annotations directly useful for driving. We test our approach using the CARLA simulator. In addition, we introduce a more challenging but realistic evaluation protocol that considers a run that reaches the destination successful only if it does not violate common traffic rules. A video summarizing this work is available at https://youtu.be/ipKAMzmJpMs , and code is available at https://github.com/twsq/sam-driving .