Empirical Risk Minimization in Non-interactive Local Differential Privacy: Efficiency and High Dimensional Case
This addresses efficiency and scalability challenges in privacy-preserving machine learning for data analysts, though it is incremental with specific improvements over prior work.
The paper tackles the Empirical Risk Minimization problem in non-interactive local differential privacy, showing that for low-dimensional cases, they avoid exponential sample complexity dependence on dimensionality, and for high-dimensional cases, they achieve error bounds based on Gaussian width rather than dimensionality.
In this paper, we study the Empirical Risk Minimization problem in the non-interactive local model of differential privacy. In the case of constant or low dimensionality ($p\ll n$), we first show that if the ERM loss function is $(\infty, T)$-smooth, then we can avoid a dependence of the sample complexity, to achieve error $α$, on the exponential of the dimensionality $p$ with base $1/α$ (i.e., $α^{-p}$), which answers a question in [smith 2017 interaction]. Our approach is based on polynomial approximation. Then, we propose player-efficient algorithms with $1$-bit communication complexity and $O(1)$ computation cost for each player. The error bound is asymptotically the same as the original one. Also with additional assumptions we show a server efficient algorithm. Next we consider the high dimensional case ($n\ll p$), we show that if the loss function is Generalized Linear function and convex, then we could get an error bound which is dependent on the Gaussian width of the underlying constrained set instead of $p$, which is lower than that in [smith 2017 interaction].