A Momentum-based Stochastic Algorithm for Linearly Constrained Nonconvex Optimization
For practitioners solving large-scale constrained nonconvex optimization problems, this method offers a simpler and more efficient alternative to existing momentum-based approaches.
This paper proposes a momentum-based augmented Lagrangian method for linearly constrained nonconvex optimization that uses only one stochastic gradient per iteration, achieving an $O(\\epsilon^{-4})$ complexity to find an $\\epsilon$-stationary solution. Experiments show improved wall-clock efficiency over recursive-momentum baselines.
This paper studies a stochastic algorithm for linearly constrained nonconvex optimization, where the objective function is smooth but only unbiased stochastic gradients with bounded variance are available. We propose a momentum-based augmented Lagrangian method that employs a Polyak-type gradient estimator and requires only one stochastic gradient evaluation per iteration. Under the standard stochastic oracle model and the smoothness condition of the expected objective, we establish a convergence guarantee in terms of the first-order KKT residual of the original constrained problem. In particular, the proposed method computes an $ε$-stationary solution in expectation within $O(ε^{-4})$ stochastic gradient evaluations. Numerical experiments further show that the proposed method achieves competitive iteration complexity and improved wall-clock efficiency compared with representative recursive-momentum baselines.