Empirical Comparison of Membership Inference Attacks in Deep Transfer Learning
This work addresses privacy leakage concerns for practitioners using transfer learning in sensitive applications, but it is incremental as it extends prior assessments by comparing more attacks.
The paper tackled the problem of evaluating privacy risks in deep transfer learning by comparing the performance of diverse membership inference attacks (MIAs), finding that attack efficacy decreases with more training data for score-based MIAs and no single MIA captures all risks, with LiRA generally performing best but IHA more effective on PatchCamelyon in high data regimes.
With the emergence of powerful large-scale foundation models, the training paradigm is increasingly shifting from from-scratch training to transfer learning. This enables high utility training with small, domain-specific datasets typical in sensitive applications. Membership inference attacks (MIAs) provide an empirical estimate of the privacy leakage by machine learning models. Yet, prior assessments of MIAs against models fine-tuned with transfer learning rely on a small subset of possible attacks. We address this by comparing performance of diverse MIAs in transfer learning settings to help practitioners identify the most efficient attacks for privacy risk evaluation. We find that attack efficacy decreases with the increase in training data for score-based MIAs. We find that there is no one MIA which captures all privacy risks in models trained with transfer learning. While the Likelihood Ratio Attack (LiRA) demonstrates superior performance across most experimental scenarios, the Inverse Hessian Attack (IHA) proves to be more effective against models fine-tuned on PatchCamelyon dataset in high data regime.