Protecting Split Learning by Potential Energy Loss
This work addresses privacy risks in split learning for applications requiring secure collaborative machine learning, presenting an incremental improvement over existing defenses.
The paper tackles privacy leakage in split learning by addressing the vulnerability of forward embeddings to label inference attacks, proposing a potential energy loss method that complicates embeddings to reduce attack effectiveness, with experiments showing significant reductions in both fine-tuning and clustering attack performance.
As a practical privacy-preserving learning method, split learning has drawn much attention in academia and industry. However, its security is constantly being questioned since the intermediate results are shared during training and inference. In this paper, we focus on the privacy leakage from the forward embeddings of split learning. Specifically, since the forward embeddings contain too much information about the label, the attacker can either use a few labeled samples to fine-tune the top model or perform unsupervised attacks such as clustering to infer the true labels from the forward embeddings. To prevent such kind of privacy leakage, we propose the potential energy loss to make the forward embeddings become more 'complicated', by pushing embeddings of the same class towards the decision boundary. Therefore, it is hard for the attacker to learn from the forward embeddings. Experiment results show that our method significantly lowers the performance of both fine-tuning attacks and clustering attacks.