Hyperparameters in Score-Based Membership Inference Attacks
This work addresses privacy leakage risks for machine learning models in transfer learning settings, but it is incremental as it builds on existing score-based MIA frameworks.
The paper tackles the problem of membership inference attacks (MIAs) in transfer learning by showing that knowledge of target model hyperparameters is not necessary, proposing a novel approach to select hyperparameters for shadow models that matches output distributions, resulting in attack performance nearly indistinguishable from using target hyperparameters, with no statistically significant increase in vulnerability when using training data for hyperparameter optimization in differentially private transfer learning.
Membership Inference Attacks (MIAs) have emerged as a valuable framework for evaluating privacy leakage by machine learning models. Score-based MIAs are distinguished, in particular, by their ability to exploit the confidence scores that the model generates for particular inputs. Existing score-based MIAs implicitly assume that the adversary has access to the target model's hyperparameters, which can be used to train the shadow models for the attack. In this work, we demonstrate that the knowledge of target hyperparameters is not a prerequisite for MIA in the transfer learning setting. Based on this, we propose a novel approach to select the hyperparameters for training the shadow models for MIA when the attacker has no prior knowledge about them by matching the output distributions of target and shadow models. We demonstrate that using the new approach yields hyperparameters that lead to an attack near indistinguishable in performance from an attack that uses target hyperparameters to train the shadow models. Furthermore, we study the empirical privacy risk of unaccounted use of training data for hyperparameter optimization (HPO) in differentially private (DP) transfer learning. We find no statistically significant evidence that performing HPO using training data would increase vulnerability to MIA.