Genetic Learning for Designing Sim-to-Real Data Augmentations
This work addresses the challenge of optimizing data augmentations for specific sim-to-real datasets in object detection, offering an automated and interpretable solution, though it is incremental as it builds on existing augmentation and genetic programming techniques.
The paper tackles the problem of selecting effective data augmentation policies for sim-to-real transfer in object detection by introducing two interpretable metrics that predict policy performance and a genetic programming method, GeneticAugment, to automatically design policies without model training, validated by strong correlation with real data performance.
Data augmentations are useful in closing the sim-to-real domain gap when training on synthetic data. This is because they widen the training data distribution, thus encouraging the model to generalize better to other domains. Many image augmentation techniques exist, parametrized by different settings, such as strength and probability. This leads to a large space of different possible augmentation policies. Some policies work better than others for overcoming the sim-to-real gap for specific datasets, and it is unclear why. This paper presents two different interpretable metrics that can be combined to predict how well a certain augmentation policy will work for a specific sim-to-real setting, focusing on object detection. We validate our metrics by training many models with different augmentation policies and showing a strong correlation with performance on real data. Additionally, we introduce GeneticAugment, a genetic programming method that can leverage these metrics to automatically design an augmentation policy for a specific dataset without needing to train a model.