Low-Cost High-Power Membership Inference Attacks
This work addresses data privacy risks in machine learning by providing a practical and accurate method for assessing if specific data points were used in training, with incremental improvements in robustness and efficiency.
The paper tackles the problem of membership inference attacks on machine learning models by designing a novel statistical test called RMIA, which achieves superior test power compared to prior methods, even at extremely low false positive rates as low as 0, with low computational overhead.
Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning.