Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Coupling
This addresses optimization efficiency for machine learning and AI researchers, offering incremental improvements in convergence rates for saddle-point problems.
The paper tackles the convex-concave saddle-point problem by proposing an Accelerated Primal-Dual Gradient Method (APDG), achieving optimal linear convergence in strongly-convex-strongly-concave cases and accelerated linear convergence in weaker convexity scenarios, with a linearly convergent algorithm for general smooth convex-concave problems without strong convexity requirements.
In this paper we study the convex-concave saddle-point problem $\min_x \max_y f(x) + y^T \mathbf{A} x - g(y)$, where $f(x)$ and $g(y)$ are smooth and convex functions. We propose an Accelerated Primal-Dual Gradient Method (APDG) for solving this problem, achieving (i) an optimal linear convergence rate in the strongly-convex-strongly-concave regime, matching the lower complexity bound (Zhang et al., 2021), and (ii) an accelerated linear convergence rate in the case when only one of the functions $f(x)$ and $g(y)$ is strongly convex or even none of them are. Finally, we obtain a linearly convergent algorithm for the general smooth and convex-concave saddle point problem $\min_x \max_y F(x,y)$ without the requirement of strong convexity or strong concavity.