NoPeek: Information leakage reduction to share activations in distributed deep learning
This addresses privacy risks for clients with sensitive data in distributed learning, though it is incremental as it builds on existing leakage reduction methods.
The paper tackled the problem of sensitive data leakage in distributed deep learning by minimizing distance correlation between raw data and intermediary representations, resulting in reduced leakage while maintaining model accuracy on image datasets.
For distributed machine learning with sensitive data, we demonstrate how minimizing distance correlation between raw data and intermediary representations reduces leakage of sensitive raw data patterns across client communications while maintaining model accuracy. Leakage (measured using distance correlation between input and intermediate representations) is the risk associated with the invertibility of raw data from intermediary representations. This can prevent client entities that hold sensitive data from using distributed deep learning services. We demonstrate that our method is resilient to such reconstruction attacks and is based on reduction of distance correlation between raw data and learned representations during training and inference with image datasets. We prevent such reconstruction of raw data while maintaining information required to sustain good classification accuracies.