Combining Adversaries with Anti-adversaries in Training
This addresses the problem of enhancing fairness and robustness in deep learning for applications like noisy or imbalanced data, though it is incremental as it builds on existing adversarial training methods.
The study investigated how adversarial training affects fairness, robustness, and generalization in deep neural networks under a more general perturbation scope, finding that combining adversaries and anti-adversaries in training can improve fairness and balance robustness and generalization in scenarios like noisy label and imbalance learning.
Adversarial training is an effective learning technique to improve the robustness of deep neural networks. In this study, the influence of adversarial training on deep learning models in terms of fairness, robustness, and generalization is theoretically investigated under more general perturbation scope that different samples can have different perturbation directions (the adversarial and anti-adversarial directions) and varied perturbation bounds. Our theoretical explorations suggest that the combination of adversaries and anti-adversaries (samples with anti-adversarial perturbations) in training can be more effective in achieving better fairness between classes and a better tradeoff between robustness and generalization in some typical learning scenarios (e.g., noisy label learning and imbalance learning) compared with standard adversarial training. On the basis of our theoretical findings, a more general learning objective that combines adversaries and anti-adversaries with varied bounds on each training sample is presented. Meta learning is utilized to optimize the combination weights. Experiments on benchmark datasets under different learning scenarios verify our theoretical findings and the effectiveness of the proposed methodology.