Image classifiers can not be made robust to small perturbations
This reveals a foundational limitation for computer vision systems, indicating that robustness to small perturbations cannot be achieved in general, which is critical for security and reliability applications.
The paper demonstrates that sensitivity to small perturbations is a fundamental property of image classifiers, showing that for any classifier over n-by-n images, it is possible to change the classification of almost all images in all but one class with perturbations of size O(n^{1/max(p,1)}) in any p-norm for p ≥ 0.
The sensitivity of image classifiers to small perturbations in the input is often viewed as a defect of their construction. We demonstrate that this sensitivity is a fundamental property of classifiers. For any arbitrary classifier over the set of $n$-by-$n$ images, we show that for all but one class it is possible to change the classification of all but a tiny fraction of the images in that class with a perturbation of size $O(n^{1/\max{(p,1)}})$ when measured in any $p$-norm for $p \geq 0$. We then discuss how this phenomenon relates to human visual perception and the potential implications for the design considerations of computer vision systems.