Adaptive Divergence for Rapid Adversarial Optimization
This addresses computational bottlenecks for researchers using heavy generators in adversarial optimization, though it appears incremental as it builds on existing AO methods.
The paper tackles the computational cost of training high-capacity models in Adversarial Optimization by introducing a novel family of divergences that vary model capacity, leading to significant acceleration in sample efficiency for tasks like tuning a physics simulator.
Adversarial Optimization (AO) provides a reliable, practical way to match two implicitly defined distributions, one of which is usually represented by a sample of real data, and the other is defined by a generator. Typically, AO involves training of a high-capacity model on each step of the optimization. In this work, we consider computationally heavy generators, for which training of high-capacity models is associated with substantial computational costs. To address this problem, we introduce a novel family of divergences, which varies the capacity of the underlying model, and allows for a significant acceleration with respect to the number of samples drawn from the generator. We demonstrate the performance of the proposed divergences on several tasks, including tuning parameters of a physics simulator, namely, Pythia event generator.