Towards Generalizing Sensorimotor Control Across Weather Conditions
This work addresses the challenge of sensorimotor control generalization for autonomous vehicles in varying weather, but it is incremental as it builds on existing teacher-student and translation methods.
The paper tackles the problem of limited labeled data for vision-based vehicle control by proposing a framework that transfers steering angle data from one weather condition to multiple others using teacher-student learning and image-to-image translation, achieving generalization across different weather conditions with only ground truth labels from one domain.
The ability of deep learning models to generalize well across different scenarios depends primarily on the quality and quantity of annotated data. Labeling large amounts of data for all possible scenarios that a model may encounter would not be feasible; if even possible. We propose a framework to deal with limited labeled training data and demonstrate it on the application of vision-based vehicle control. We show how limited steering angle data available for only one condition can be transferred to multiple different weather scenarios. This is done by leveraging unlabeled images in a teacher-student learning paradigm complemented with an image-to-image translation network. The translation network transfers the images to a new domain, whereas the teacher provides soft supervised targets to train the student on this domain. Furthermore, we demonstrate how utilization of auxiliary networks can reduce the size of a model at inference time, without affecting the accuracy. The experiments show that our approach generalizes well across multiple different weather conditions using only ground truth labels from one domain.