Learning To Simulate
This addresses data scarcity issues for machine learning practitioners by automating simulator tuning, though it is incremental as it builds on existing simulation methods.
The paper tackles the problem of costly or hard-to-acquire training data by proposing a reinforcement learning method to automatically adjust simulator parameters to maximize model accuracy, finding it quickly converges to optimal parameters and discovers good sets for an image rendering simulator.
Simulation is a useful tool in situations where training data for machine learning models is costly to annotate or even hard to acquire. In this work, we propose a reinforcement learning-based method for automatically adjusting the parameters of any (non-differentiable) simulator, thereby controlling the distribution of synthesized data in order to maximize the accuracy of a model trained on that data. In contrast to prior art that hand-crafts these simulation parameters or adjusts only parts of the available parameters, our approach fully controls the simulator with the actual underlying goal of maximizing accuracy, rather than mimicking the real data distribution or randomly generating a large volume of data. We find that our approach (i) quickly converges to the optimal simulation parameters in controlled experiments and (ii) can indeed discover good sets of parameters for an image rendering simulator in actual computer vision applications.