Approximate Last Iterate Convergence in Overparameterized GANs
This addresses convergence issues in GAN training for machine learning practitioners, but it is incremental as it builds on prior methods.
The paper tackled the problem of last iterate convergence in overparameterized GANs, showing that Implicit Update and Predictive Methods dynamics converge to a neighborhood around the optimum, with the neighborhood size shrinking as network width increases, in contrast to prior average iterate convergence guarantees.
In this work, we showed that the Implicit Update and Predictive Methods dynamics introduced in prior work satisfy last iterate convergence to a neighborhood around the optimum in overparameterized GANs, where the size of the neighborhood shrinks with the width of the neural network. This is in contrast to prior results, which only guaranteed average iterate convergence.