Forward-backward-forward methods with variance reduction for stochastic variational inequalities
This work addresses computational efficiency in optimization for machine learning and related fields, though it appears incremental as it extends an existing deterministic method to stochastic settings with variance reduction.
The authors tackled the problem of solving pseudo-monotone stochastic variational inequalities by developing a new stochastic algorithm with variance reduction, building on Tseng's forward-backward-forward method, which achieves almost sure convergence to an optimal solution using only a single projection per iteration.
We develop a new stochastic algorithm with variance reduction for solving pseudo-monotone stochastic variational inequalities. Our method builds on Tseng's forward-backward-forward (FBF) algorithm, which is known in the deterministic literature to be a valuable alternative to Korpelevich's extragradient method when solving variational inequalities over a convex and closed set governed by pseudo-monotone, Lipschitz continuous operators. The main computational advantage of Tseng's algorithm is that it relies only on a single projection step and two independent queries of a stochastic oracle. Our algorithm incorporates a variance reduction mechanism and leads to almost sure (a.s.) convergence to an optimal solution. To the best of our knowledge, this is the first stochastic look-ahead algorithm achieving this by using only a single projection at each iteration..