One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks
This addresses data privacy concerns for data owners by creating unlearnable datasets that resist common defenses like adversarial training, though it is incremental in improving robustness over prior unlearnable example methods.
The paper tackles the problem of protecting data from unauthorized deep neural network training by introducing a method that perturbs only one pixel per image, degrading model accuracy to near-untrained levels, with results showing only 10.61% accuracy on CIFAR-10-S compared to 83.02% for existing methods under adversarial training.
Unlearnable examples (ULEs) aim to protect data from unauthorized usage for training DNNs. Existing work adds $\ell_\infty$-bounded perturbations to the original sample so that the trained model generalizes poorly. Such perturbations, however, are easy to eliminate by adversarial training and data augmentations. In this paper, we resolve this problem from a novel perspective by perturbing only one pixel in each image. Interestingly, such a small modification could effectively degrade model accuracy to almost an untrained counterpart. Moreover, our produced \emph{One-Pixel Shortcut (OPS)} could not be erased by adversarial training and strong augmentations. To generate OPS, we perturb in-class images at the same position to the same target value that could mostly and stably deviate from all the original images. Since such generation is only based on images, OPS needs significantly less computation cost than the previous methods using DNN generators. Based on OPS, we introduce an unlearnable dataset called CIFAR-10-S, which is indistinguishable from CIFAR-10 by humans but induces the trained model to extremely low accuracy. Even under adversarial training, a ResNet-18 trained on CIFAR-10-S has only 10.61% accuracy, compared to 83.02% by the existing error-minimizing method.