Differentially Private Empirical Risk Minimization Revisited: Faster and More General
This work addresses privacy-preserving machine learning for data analysts, offering incremental improvements in efficiency and generality for ERM problems.
The paper tackles differentially private empirical risk minimization (ERM) by developing faster algorithms with improved utility bounds for various settings, including smooth convex and non-convex loss functions, achieving reduced gradient complexity compared to prior work.
In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared with previous work. For ERM with smooth convex loss function in high-dimensional ($p\gg n$) setting, we give an algorithm which achieves the upper bound with less gradient complexity than previous ones. At last, we generalize the expected excess empirical risk from convex loss functions to non-convex ones satisfying the Polyak-Lojasiewicz condition and give a tighter upper bound on the utility than the one in \cite{ijcai2017-548}.