Stability and Generalization of Adversarial Diffusion Training
This work addresses generalization issues in adversarial training for decentralized networks, but it is incremental as it extends known single-agent results to a decentralized setting.
The authors tackled the problem of robust overfitting and generalization gap in adversarial training by analyzing its stability and generalization under a diffusion strategy for convex losses. They derived a bound showing generalization error grows with adversarial perturbation strength and training steps, validated by numerical experiments on logistic regression.
Algorithmic stability is an established tool for analyzing generalization. While adversarial training enhances model robustness, it often suffers from robust overfitting and an enlarged generalization gap. Although recent work has established the convergence of adversarial training in decentralized networks, its generalization properties remain unexplored. This work presents a stability-based generalization analysis of adversarial training under the diffusion strategy for convex losses. We derive a bound showing that the generalization error grows with both the adversarial perturbation strength and the number of training steps, a finding consistent with single-agent case but novel for decentralized settings. Numerical experiments on logistic regression validate these theoretical predictions.