Lossy Compression of Noisy Data for Private and Data-Efficient Learning
This work addresses storage-efficient and privacy-preserving learning for sensitive user data, representing an incremental improvement by integrating noise injection with compression techniques.
The paper tackles the problem of reducing storage costs and ensuring privacy for sensitive user data in machine learning by proposing a framework that combines noise injection and lossy compression, showing that it maintains learning utility while achieving substantial storage reduction and privacy protection, with experimental results on the CelebA dataset indicating no essential loss in classification accuracy and improved robustness against adversarial data.
Storage-efficient privacy-preserving learning is crucial due to increasing amounts of sensitive user data required for modern learning tasks. We propose a framework for reducing the storage cost of user data while at the same time providing privacy guarantees, without essential loss in the utility of the data for learning. Our method comprises noise injection followed by lossy compression. We show that, when appropriately matching the lossy compression to the distribution of the added noise, the compressed examples converge, in distribution, to that of the noise-free training data as the sample size of the training data (or the dimension of the training data) increases. In this sense, the utility of the data for learning is essentially maintained, while reducing storage and privacy leakage by quantifiable amounts. We present experimental results on the CelebA dataset for gender classification and find that our suggested pipeline delivers in practice on the promise of the theory: the individuals in the images are unrecognizable (or less recognizable, depending on the noise level), overall storage of the data is substantially reduced, with no essential loss (and in some cases a slight boost) to the classification accuracy. As an added bonus, our experiments suggest that our method yields a substantial boost to robustness in the face of adversarial test data.