A Constant Step Stochastic Douglas-Rachford Algorithm with Application to Non Separable Regularizations
This work addresses optimization challenges in machine learning, particularly for structured regularization, but is incremental as it extends an existing algorithm to a stochastic setting.
The paper tackles the problem of minimizing a sum of two convex functions by proposing a stochastic version of the Douglas-Rachford algorithm with constant step size, establishing that iterates stay close to the solution set with high probability for small step sizes.
The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions. The algorithm consists in fixed point iterations involving computations of the proximity operators of the two functions separately. The paper investigates a stochastic version of the algorithm where both functions are random and the step size is constant. We establish that the iterates of the algorithm stay close to the set of solution with high probability when the step size is small enough. Application to structured regularization is considered.