Meta-simulation for the Automated Design of Synthetic Overhead Imagery
This addresses the problem of efficient synthetic data generation for domain-specific applications like overhead imagery, though it builds incrementally on prior meta-simulation approaches.
The paper tackles the challenge of automatically designing synthetic overhead imagery for training machine learning models when real-world data is unlabeled, proposing Neural-Adjoint Meta-Simulation (NAMS) which infers designs that match target imagery and yields superior segmentation results compared to existing methods.
The use of synthetic (or simulated) data for training machine learning models has grown rapidly in recent years. Synthetic data can often be generated much faster and more cheaply than its real-world counterpart. One challenge of using synthetic imagery however is scene design: e.g., the choice of content and its features and spatial arrangement. To be effective, this design must not only be realistic, but appropriate for the target domain, which (by assumption) is unlabeled. In this work, we propose an approach to automatically choose the design of synthetic imagery based upon unlabeled real-world imagery. Our approach, termed Neural-Adjoint Meta-Simulation (NAMS), builds upon the seminal recent meta-simulation approaches. In contrast to the current state-of-the-art methods, our approach can be pre-trained once offline, and then provides fast design inference for new target imagery. Using both synthetic and real-world problems, we show that NAMS infers synthetic designs that match both the in-domain and out-of-domain target imagery, and that training segmentation models with NAMS-designed imagery yields superior results compared to naïve randomized designs and state-of-the-art meta-simulation methods.