Active manifolds, stratifications, and convergence to local minima in nonsmooth optimization
This provides theoretical guarantees for optimization algorithms in nonsmooth settings, which is incremental but important for applications in machine learning and engineering.
The paper tackles the problem of ensuring convergence to local minimizers in nonsmooth optimization by analyzing the subgradient method applied to generic Lipschitz continuous and subdifferentially regular functions definable in o-minimal structures, showing that it converges only to local minimizers through an interpretation as an approximate Riemannian gradient method on a distinguished submanifold.
We show that the subgradient method converges only to local minimizers when applied to generic Lipschitz continuous and subdifferentially regular functions that are definable in an o-minimal structure. At a high level, the argument we present is appealingly transparent: we interpret the nonsmooth dynamics as an approximate Riemannian gradient method on a certain distinguished submanifold that captures the nonsmooth activity of the function. In the process, we develop new regularity conditions in nonsmooth analysis that parallel the stratification conditions of Whitney, Kuo, and Verdier and extend stochastic processes techniques of Pemantle.