Min-Max Optimization Is Strictly Easier Than Variational Inequalities
This work addresses a foundational optimization problem for machine learning and AI researchers, offering incremental theoretical insights into algorithm efficiency.
The paper tackles the problem of solving convex-concave min-max problems by investigating whether bypassing the reduction to variational inequalities can lead to faster algorithms, and shows that in unconstrained quadratic settings, first-order algorithms achieve strictly better optimal convergence rates for min-max problems compared to variational inequalities.
Classically, a mainstream approach for solving a convex-concave min-max problem is to instead solve the variational inequality problem arising from its first-order optimality conditions. Is it possible to solve min-max problems faster by bypassing this reduction? This paper initiates this investigation. We show that the answer is yes in the textbook setting of unconstrained quadratic objectives: the optimal convergence rate for first-order algorithms is strictly better for min-max problems than for the corresponding variational inequalities. The key reason that min-max algorithms can be faster is that they can exploit the asymmetry of the min and max variables--a property that is lost in the reduction to variational inequalities. Central to our analyses are sharp characterizations of optimal convergence rates in terms of extremal polynomials which we compute using Green's functions and conformal mappings.