Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness
This work addresses adversarial robustness for deep learning models, offering a novel training approach that is incremental but provides specific gains in defense against perturbations.
The paper tackles the problem of adversarial robustness in deep classifiers by analyzing decision boundary dynamics during training, revealing that existing methods like Adversarial Training can inadvertently decrease margins for some points. It proposes Dynamics-aware Robust Training (DyART), which directly optimizes margins to alleviate conflicting dynamics, achieving improved robustness on CIFAR-10 and Tiny-ImageNet datasets compared to state-of-the-art defenses.
The robustness of a deep classifier can be characterized by its margins: the decision boundary's distances to natural data points. However, it is unclear whether existing robust training methods effectively increase the margin for each vulnerable point during training. To understand this, we propose a continuous-time framework for quantifying the relative speed of the decision boundary with respect to each individual point. Through visualizing the moving speed of the decision boundary under Adversarial Training, one of the most effective robust training algorithms, a surprising moving-behavior is revealed: the decision boundary moves away from some vulnerable points but simultaneously moves closer to others, decreasing their margins. To alleviate these conflicting dynamics of the decision boundary, we propose Dynamics-aware Robust Training (DyART), which encourages the decision boundary to engage in movement that prioritizes increasing smaller margins. In contrast to prior works, DyART directly operates on the margins rather than their indirect approximations, allowing for more targeted and effective robustness improvement. Experiments on the CIFAR-10 and Tiny-ImageNet datasets verify that DyART alleviates the conflicting dynamics of the decision boundary and obtains improved robustness under various perturbation sizes compared to the state-of-the-art defenses. Our code is available at https://github.com/Yuancheng-Xu/Dynamics-Aware-Robust-Training.