Separable Approximations and Decomposition Methods for the Augmented Lagrangian
This work provides theoretical insights into optimization algorithms for separable approximations, which is incremental for researchers in convex optimization and decomposition methods.
The paper compares two decomposition methods, DQAM and PCDM, for minimizing the augmented Lagrangian, showing they are equivalent for feasibility problems and proving an improved complexity bound for PCDM that is at least 8(L'/L̄)(ω-1)^2 times better than DQAM under strong convexity.
In this paper we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian. In particular, we study and compare the Diagonal Quadratic Approximation Method (DQAM) of Mulvey and Ruszczyński and the Parallel Coordinate Descent Method (PCDM) of Richtárik and Takáč. We show that the two methods are equivalent for feasibility problems up to the selection of a single step-size parameter. Furthermore, we prove an improved complexity bound for PCDM under strong convexity, and show that this bound is at least $8(L'/\bar{L})(ω-1)^2$ times better than the best known bound for DQAM, where $ω$ is the degree of partial separability and $L'$ and $\bar{L}$ are the maximum and average of the block Lipschitz constants of the gradient of the quadratic penalty appearing in the augmented Lagrangian.