Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation
This work provides a theoretical foundation for understanding adversarial training in machine learning, though it is incremental as it builds on existing methods with a new analytical approach.
The paper tackled the problem of analyzing adversarial training for gradient-based models by establishing a continuous-time approximation, which allowed for precise theoretical comparisons and confirmed robustness from a gradient-flow viewpoint, supported by analytical and numerical examples.
Adversarial training has gained great popularity as one of the most effective defenses for deep neural network and more generally for gradient-based machine learning models against adversarial perturbations on data points. This paper establishes a continuous-time approximation for the mini-max game of adversarial training. This approximation approach allows for precise and analytical comparisons between stochastic gradient descent and its adversarial training counterpart; and confirms theoretically the robustness of adversarial training from a new gradient-flow viewpoint. The analysis is then corroborated through various analytical and numerical examples.