Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generator
This addresses privacy concerns in distributed learning for applications like healthcare or finance, but it is incremental as it builds on existing FL and knowledge distillation techniques.
The paper tackles privacy leakage risks in Federated Learning by proposing FedMD-CG, a method that uses a conditional generator and knowledge distillation to achieve high-level privacy preservation while maintaining competitive performance, with experimental validation on image classification tasks showing strong results.
Federated Learning (FL) is gaining popularity as a distributed learning framework that only shares model parameters or gradient updates and keeps private data locally. However, FL is at risk of privacy leakage caused by privacy inference attacks. And most existing privacy-preserving mechanisms in FL conflict with achieving high performance and efficiency. Therefore, we propose FedMD-CG, a novel FL method with highly competitive performance and high-level privacy preservation, which decouples each client's local model into a feature extractor and a classifier, and utilizes a conditional generator instead of the feature extractor to perform server-side model aggregation. To ensure the consistency of local generators and classifiers, FedMD-CG leverages knowledge distillation to train local models and generators at both the latent feature level and the logit level. Also, we construct additional classification losses and design new diversity losses to enhance client-side training. FedMD-CG is robust to data heterogeneity and does not require training extra discriminators (like cGAN). We conduct extensive experiments on various image classification tasks to validate the superiority of FedMD-CG.