A general approach to bridge the reality-gap
This addresses the costly data collection issue for deploying ML models in real-world scenarios, though it is incremental as it builds on domain adaptation and transfer learning methods.
The paper tackles the problem of machine learning models failing to transfer from canonical datasets to real-world distributions by proposing an unsupervised transformation that brings arbitrary images towards a canonical distribution, recovering half of the performance loss on distorted data and validating it on real-world printed and photographed images.
Employing machine learning models in the real world requires collecting large amounts of data, which is both time consuming and costly to collect. A common approach to circumvent this is to leverage existing, similar data-sets with large amounts of labelled data. However, models trained on these canonical distributions do not readily transfer to real-world ones. Domain adaptation and transfer learning are often used to breach this "reality gap", though both require a substantial amount of real-world data. In this paper we discuss a more general approach: we propose learning a general transformation to bring arbitrary images towards a canonical distribution where we can naively apply the trained machine learning models. This transformation is trained in an unsupervised regime, leveraging data augmentation to generate off-canonical examples of images and training a Deep Learning model to recover their original counterpart. We quantify the performance of this transformation using pre-trained ImageNet classifiers, demonstrating that this procedure can recover half of the loss in performance on the distorted data-set. We then validate the effectiveness of this approach on a series of pre-trained ImageNet models on a real world data set collected by printing and photographing images in different lighting conditions.