Regularized Loss Minimizers with Local Data Perturbation: Consistency and Data Irrecoverability
This work addresses privacy concerns for data owners in machine learning by providing a theoretical framework that allows for data perturbation without significantly compromising learning performance, though it appears incremental in building on existing regularization methods.
The paper tackles the problem of ensuring data privacy in machine learning by introducing data irrecoverability, showing that it is a weaker condition than traditional privacy, and demonstrates that regularized loss minimizers with perturbed data maintain generalization guarantees with convergence rates only increased by a constant factor related to the perturbation.
We introduce a new concept, data irrecoverability, and show that the well-studied concept of data privacy is sufficient but not necessary for data irrecoverability. We show that there are several regularized loss minimization problems that can use perturbed data with theoretical guarantees of generalization, i.e., loss consistency. Our results quantitatively connect the convergence rates of the learning problems to the impossibility for any adversary for recovering the original data from perturbed observations. In addition, we show several examples where the convergence rates with perturbed data only increase the convergence rates with original data within a constant factor related to the amount of perturbation, i.e., noise.