Can sparsity improve the privacy of neural networks?
This addresses privacy concerns in machine learning for data-sensitive applications, but it is incremental as it critiques prior work and highlights methodological caveats.
The paper investigates whether sparsity in neural networks improves data privacy during training, finding positive correlations between sparsity, privacy, and classification error, but warns that comparing models with different sparsity levels can be misleading due to error correlations.
Sparse neural networks are mainly motivated by ressource efficiency since they use fewer parameters than their dense counterparts but still reach comparable accuracies. This article empirically investigates whether sparsity could also improve the privacy of the data used to train the networks. The experiments show positive correlations between the sparsity of the model, its privacy, and its classification error. Simply comparing the privacy of two models with different sparsity levels can yield misleading conclusions on the role of sparsity, because of the additional correlation with the classification error. From this perspective, some caveats are raised about previous works that investigate sparsity and privacy.