Differentially Private Optimization for Smooth Nonconvex ERM
This work addresses privacy-preserving optimization for nonconvex ERM, which is incremental as it builds on existing methods with improvements in speed and practicality.
The paper tackles the problem of differentially private optimization for smooth nonconvex empirical risk minimization (ERM) by developing algorithms that find approximate second-order solutions, with numerical experiments demonstrating their effectiveness.
We develop simple differentially private optimization algorithms that move along directions of (expected) descent to find an approximate second-order solution for nonconvex ERM. We use line search, mini-batching, and a two-phase strategy to improve the speed and practicality of the algorithm. Numerical experiments demonstrate the effectiveness of these approaches.