Learning from aggregated data with a maximum entropy model
This enables machine learning on privacy-protected aggregated data, which is important for applications where raw data cannot be shared due to privacy constraints.
The authors tackled the problem of training machine learning classifiers from aggregated data (a common output of differential privacy mechanisms) by developing a maximum entropy model that approximates the unobserved feature distribution. They showed empirically that this model achieves performance comparable to logistic regression trained on full unaggregated data across several public datasets.
Aggregating a dataset, then injecting some noise, is a simple and common way to release differentially private data.However, aggregated data -- even without noise -- is not an appropriate input for machine learning classifiers.In this work, we show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. The resulting model is a Markov Random Field (MRF), and we detail how to apply, modify and scale a MRF training algorithm to our setting. Finally we present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.