Generative Models for Effective ML on Private, Decentralized Datasets
This work addresses the challenge of data inspection for modelers in privacy-sensitive and federated learning settings, offering a novel solution that is incremental in applying existing methods to new constraints.
The paper tackles the problem of debugging data issues in privacy-sensitive and decentralized datasets by using generative models trained with federated learning and differential privacy, enabling effective identification and resolution of common data problems without direct data inspection.
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact. Manual inspection of raw data - of representative samples, of outliers, of misclassifications - is an essential tool in a) identifying and fixing problems in the data, b) generating new modeling hypotheses, and c) assigning or refining human-provided labels. However, manual data inspection is problematic for privacy sensitive datasets, such as those representing the behavior of real-world individuals. Furthermore, manual data inspection is impossible in the increasingly important setting of federated learning, where raw examples are stored at the edge and the modeler may only access aggregated outputs such as metrics or model parameters. This paper demonstrates that generative models - trained using federated methods and with formal differential privacy guarantees - can be used effectively to debug many commonly occurring data issues even when the data cannot be directly inspected. We explore these methods in applications to text with differentially private federated RNNs and to images using a novel algorithm for differentially private federated GANs.