Knowledge Cross-Distillation for Membership Privacy
This addresses privacy risks in sensitive domains like medicine and finance where public data is scarce, offering an incremental improvement over existing defenses.
The authors tackled the problem of defending against membership inference attacks (MIAs) without requiring public data, proposing a novel defense using knowledge distillation that achieves comparable privacy protection and accuracy to the state-of-the-art method on tabular datasets and a better privacy-utility trade-off on an image dataset.
A membership inference attack (MIA) poses privacy risks for the training data of a machine learning model. With an MIA, an attacker guesses if the target data are a member of the training dataset. The state-of-the-art defense against MIAs, distillation for membership privacy (DMP), requires not only private data for protection but a large amount of unlabeled public data. However, in certain privacy-sensitive domains, such as medicine and finance, the availability of public data is not guaranteed. Moreover, a trivial method for generating public data by using generative adversarial networks significantly decreases the model accuracy, as reported by the authors of DMP. To overcome this problem, we propose a novel defense against MIAs that uses knowledge distillation without requiring public data. Our experiments show that the privacy protection and accuracy of our defense are comparable to those of DMP for the benchmark tabular datasets used in MIA research, Purchase100 and Texas100, and our defense has a much better privacy-utility trade-off than those of the existing defenses that also do not use public data for the image dataset CIFAR10.