Omega: Optimistic EMA Gradients
This addresses noise issues in stochastic game optimization for machine learning applications, but it is incremental as it builds on existing optimistic gradient methods without convergence guarantees.
The paper tackles the problem of noise sensitivity and convergence failure in stochastic min-max optimization for GANs and adversarial training by introducing Omega, a method that uses optimistic-like updates with an EMA of historic gradients. Experiments show it outperforms the optimistic gradient method on stochastic games with linear players.
Stochastic min-max optimization has gained interest in the machine learning community with the advancements in GANs and adversarial training. Although game optimization is fairly well understood in the deterministic setting, some issues persist in the stochastic regime. Recent work has shown that stochastic gradient descent-ascent methods such as the optimistic gradient are highly sensitive to noise or can fail to converge. Although alternative strategies exist, they can be prohibitively expensive. We introduce Omega, a method with optimistic-like updates that mitigates the impact of noise by incorporating an EMA of historic gradients in its update rule. We also explore a variation of this algorithm that incorporates momentum. Although we do not provide convergence guarantees, our experiments on stochastic games show that Omega outperforms the optimistic gradient method when applied to linear players.