Non-stationary Douglas-Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence
This work addresses a practical bottleneck in optimization algorithms for researchers and practitioners, offering an incremental improvement over classical methods.
The paper tackles the problem of stepsize tuning in the Douglas-Rachford method for optimization by developing an adaptive stepsize rule, proving convergence for non-stationary schemes and demonstrating efficiency in numerical examples.
We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at developing an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for stepsize tuning. We analyze a general non-stationary DR scheme and prove its convergence for a convergent sequence of stepsizes with summable increments. This, in turn, proves the convergence of the method with the new adaptive stepsize rule. We also derive the related non-stationary alternating direction method of multipliers (ADMM) from such a non-stationary DR method. We illustrate the efficiency of the proposed methods on several numerical examples.