No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"
This work highlights critical flaws in privacy claims for ML methods, which is important for researchers and practitioners to avoid false security in privacy-preserving techniques.
The paper critiques a prior ICML 2022 award-winning work that claimed dataset condensation improves data privacy in ML training, arguing that flaws in its evaluation and analysis invalidate this claim and noting that standard methods like DP-SGD offer better privacy and accuracy.
New methods designed to preserve data privacy require careful scrutiny. Failure to preserve privacy is hard to detect, and yet can lead to catastrophic results when a system implementing a ``privacy-preserving'' method is attacked. A recent work selected for an Outstanding Paper Award at ICML 2022 (Dong et al., 2022) claims that dataset condensation (DC) significantly improves data privacy when training machine learning models. This claim is supported by theoretical analysis of a specific dataset condensation technique and an empirical evaluation of resistance to some existing membership inference attacks. In this note we examine the claims in the work of Dong et al. (2022) and describe major flaws in the empirical evaluation of the method and its theoretical analysis. These flaws imply that their work does not provide statistically significant evidence that DC improves the privacy of training ML models over a naive baseline. Moreover, previously published results show that DP-SGD, the standard approach to privacy preserving ML, simultaneously gives better accuracy and achieves a (provably) lower membership attack success rate.