Revisiting Membership Inference Under Realistic Assumptions
This work addresses privacy risks in machine learning for data owners and practitioners, but it is incremental as it builds on existing membership inference research.
The paper tackles membership inference attacks under more realistic assumptions, such as skewed priors and adversary-defined thresholds, by developing a new attack that uses loss function minima and achieves high positive predictive value (PPV) where previous methods fail.
We study membership inference in settings where some of the assumptions typically used in previous research are relaxed. First, we consider skewed priors, to cover cases such as when only a small fraction of the candidate pool targeted by the adversary are actually members and develop a PPV-based metric suitable for this setting. This setting is more realistic than the balanced prior setting typically considered by researchers. Second, we consider adversaries that select inference thresholds according to their attack goals and develop a threshold selection procedure that improves inference attacks. Since previous inference attacks fail in imbalanced prior setting, we develop a new inference attack based on the intuition that inputs corresponding to training set members will be near a local minimum in the loss function, and show that an attack that combines this with thresholds on the per-instance loss can achieve high PPV even in settings where other attacks appear to be ineffective. Code for our experiments can be found here: https://github.com/bargavj/EvaluatingDPML.