BLIA: Detect model memorization in binary classification model through passive Label Inference attack
This work addresses privacy and generalization issues in machine learning by showing that models memorize training labels, which is an incremental finding as it builds on existing memorization concerns.
The paper tackles the problem of label memorization in binary classification models by introducing two passive label inference attacks (BLIA) that use model outputs like confidence scores to detect memorization, achieving success rates over 50% even with label differential privacy applied.
Model memorization has implications for both the generalization capacity of machine learning models and the privacy of their training data. This paper investigates label memorization in binary classification models through two novel passive label inference attacks (BLIA). These attacks operate passively, relying solely on the outputs of pre-trained models, such as confidence scores and log-loss values, without interacting with or modifying the training process. By intentionally flipping 50% of the labels in controlled subsets, termed "canaries," we evaluate the extent of label memorization under two conditions: models trained without label differential privacy (Label-DP) and those trained with randomized response-based Label-DP. Despite the application of varying degrees of Label-DP, the proposed attacks consistently achieve success rates exceeding 50%, surpassing the baseline of random guessing and conclusively demonstrating that models memorize training labels, even when these labels are deliberately uncorrelated with the features.