Adaptive ADMM with Spectral Penalty Parameter Selection
This addresses a reliability and automation problem for non-expert users of ADMM in constrained optimization.
The paper tackled the sensitivity of ADMM to penalty parameters by proposing an adaptive tuning method, resulting in the AADMM algorithm that achieves fast convergence and insensitivity to initial stepsize and scaling.
The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems, with differentiable or non-differentiable objective functions. Unfortunately, its performance is highly sensitive to a penalty parameter, which makes ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposing a method to adaptively tune the penalty parameters to achieve fast convergence. The resulting adaptive ADMM (AADMM) algorithm, inspired by the successful Barzilai-Borwein spectral method for gradient descent, yields fast convergence and relative insensitivity to the initial stepsize and problem scaling.