Enhancing Adversarial Training with Feature Separability
This work addresses a critical issue in adversarial robustness for deep learning models, offering a novel approach to enhance feature separability, though it is incremental in nature.
The paper tackles the problem of adversarial training in deep neural networks failing to learn effective feature representations, leading to overfitting and poor generalization. The proposed ATFS method significantly improves both clean and robust performance, as demonstrated through comprehensive experiments.
Deep Neural Network (DNN) are vulnerable to adversarial attacks. As a countermeasure, adversarial training aims to achieve robustness based on the min-max optimization problem and it has shown to be one of the most effective defense strategies. However, in this work, we found that compared with natural training, adversarial training fails to learn better feature representations for either clean or adversarial samples, which can be one reason why adversarial training tends to have severe overfitting issues and less satisfied generalize performance. Specifically, we observe two major shortcomings of the features learned by existing adversarial training methods:(1) low intra-class feature similarity; and (2) conservative inter-classes feature variance. To overcome these shortcomings, we introduce a new concept of adversarial training graph (ATG) with which the proposed adversarial training with feature separability (ATFS) enables to coherently boost the intra-class feature similarity and increase inter-class feature variance. Through comprehensive experiments, we demonstrate that the proposed ATFS framework significantly improves both clean and robust performance.