Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization
This work addresses convergence guarantees for stochastic optimization methods in nonconvex settings, which is incremental as it extends existing theory to more general conditions.
The paper tackles the problem of ensuring convergence for stochastic subgradient methods in nonsmooth nonconvex optimization by proving global stability under conditions like coercive Lyapunov functions and controlled noise, with results showing uniform boundedness and asymptotic stabilization around stable sets.
In this paper, we focus on providing convergence guarantees for stochastic subgradient methods in minimizing nonsmooth nonconvex functions. We first investigate the global stability of a general framework for stochastic subgradient methods, where the corresponding differential inclusion admits a coercive Lyapunov function. We prove that, for any sequence of sufficiently small stepsizes and approximation parameters, coupled with sufficiently controlled noises, the iterates are uniformly bounded and asymptotically stabilize around the stable set of its corresponding differential inclusion. Moreover, we develop an improved analysis to apply our proposed framework to establish the global stability of a wide range of stochastic subgradient methods, where the corresponding Lyapunov functions are possibly non-coercive. These theoretical results illustrate the promising potential of our proposed framework for establishing the global stability of various stochastic subgradient methods.