Robust and Information-theoretically Safe Bias Classifier against Adversarial Attacks
This work addresses the security of machine learning models against adversarial attacks, proposing a novel classifier that is theoretically safe, though it may be incremental in its application to specific attack types.
The authors tackled the problem of adversarial attacks on deep neural networks by introducing a bias classifier that uses only the bias parts of a ReLU network, which has zero gradient and thus resists gradient-based attacks. They proved its existence, provided a training method, and showed that adding a random component makes it information-theoretically safe, with experiments indicating it is more robust than similarly sized DNNs in most cases.
In this paper, the bias classifier is introduced, that is, the bias part of a DNN with Relu as the activation function is used as a classifier. The work is motivated by the fact that the bias part is a piecewise constant function with zero gradient and hence cannot be directly attacked by gradient-based methods to generate adversaries, such as FGSM. The existence of the bias classifier is proved and an effective training method for the bias classifier is given. It is proved that by adding a proper random first-degree part to the bias classifier, an information-theoretically safe classifier against the original-model gradient attack is obtained in the sense that the attack will generate a totally random attacking direction. This seems to be the first time that the concept of information-theoretically safe classifier is proposed. Several attack methods for the bias classifier are proposed and numerical experiments are used to show that the bias classifier is more robust than DNNs with similar size against these attacks in most cases.