Scalable Peaceman-Rachford Splitting Method with Proximal Terms
This provides a scalable stochastic optimization method for large-scale splitting problems, but it is incremental as it adapts PRSM to a stochastic setting.
The paper tackles the lack of stochastic algorithms for the Peaceman-Rachford Splitting Method (PRSM) by proposing SS-PRSM, which achieves an O(1/K) convergence rate, matching newer stochastic ADMM-based methods and outperforming general Stochastic ADMM with O(1/√K).
Along with developing of Peaceman-Rachford Splittling Method (PRSM), many batch algorithms based on it have been studied very deeply. But almost no algorithm focused on the performance of stochastic version of PRSM. In this paper, we propose a new stochastic algorithm based on PRSM, prove its convergence rate in ergodic sense, and test its performance on both artificial and real data. We show that our proposed algorithm, Stochastic Scalable PRSM (SS-PRSM), enjoys the $O(1/K)$ convergence rate, which is the same as those newest stochastic algorithms that based on ADMM but faster than general Stochastic ADMM (which is $O(1/\sqrt{K})$). Our algorithm also owns wide flexibility, outperforms many state-of-the-art stochastic algorithms coming from ADMM, and has low memory cost in large-scale splitting optimization problems.