Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems
This work offers theoretical tools for analyzing nonsmooth solutions in optimization, which is incremental but useful for researchers in mathematical optimization and machine learning.
The paper provides sufficient conditions for solutions to parametric monotone inclusion problems to be path differentiable, with formulas for computing generalized gradients, making them differentiable almost everywhere. The approach is compatible with automatic differentiation and applies to composite problems like strongly convex optimization and min-max problems.
We leverage path differentiability and a recent result on nonsmooth implicit differentiation calculus to give sufficient conditions ensuring that the solution to a monotone inclusion problem will be path differentiable, with formulas for computing its generalized gradient. A direct consequence of our result is that these solutions happen to be differentiable almost everywhere. Our approach is fully compatible with automatic differentiation and comes with assumptions which are easy to check, roughly speaking: semialgebraicity and strong monotonicity. We illustrate the scope of our results by considering three fundamental composite problem settings: strongly convex problems, dual solutions to convex minimization problems and primal-dual solutions to min-max problems.