Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints
This provides faster optimization for constrained problems like personalized PageRank, though it is incremental as it builds on existing primal-dual methods.
The paper tackles the problem of minimizing a convex function subject to strongly convex constraints, improving the complexity bound from O(1/ε) to O(1/√ε) by developing novel techniques to estimate the Lagrangian's strong convexity, which matches the lower bound and is demonstrated on Google's personalized PageRank problem.
In this paper, we introduce faster accelerated primal-dual algorithms for minimizing a convex function subject to strongly convex function constraints. Prior to our work, the best complexity bound was $\mathcal{O}(1/{\varepsilon})$, regardless of the strong convexity of the constraint function. It is unclear whether the strong convexity assumption can enable even better convergence results. To address this issue, we have developed novel techniques to progressively estimate the strong convexity of the Lagrangian function. Our approach, for the first time, effectively leverages the constraint strong convexity, obtaining an improved complexity of $\mathcal{O}(1/\sqrt{\varepsilon})$. This rate matches the complexity lower bound for strongly-convex-concave saddle point optimization and is therefore order-optimal. We show the superior performance of our methods in sparsity-inducing constrained optimization, notably Google's personalized PageRank problem. Furthermore, we show that a restarted version of the proposed methods can effectively identify the optimal solution's sparsity pattern within a finite number of steps, a result that appears to have independent significance.