Is Supervised Learning With Adversarial Features Provably Better Than Sole Supervision?
This work addresses a fundamental issue in machine learning by providing theoretical justification for combining adversarial and supervised learning, which could benefit researchers and practitioners in improving model training efficiency.
The paper tackles the problem of vanishing gradients in supervised learning near optimal regions by theoretically analyzing whether adding adversarial features from GANs can improve performance over sole supervision. It proves that under mild assumptions, supervised learning with adversarial features yields better expected empirical risk and faster convergence rates.
Generative Adversarial Networks (GAN) have shown promising results on a wide variety of complex tasks. Recent experiments show adversarial training provides useful gradients to the generator that helps attain better performance. In this paper, we intend to theoretically analyze whether supervised learning with adversarial features can outperform sole supervision, or not. First, we show that supervised learning without adversarial features suffer from vanishing gradient issue in near optimal region. Second, we analyze how adversarial learning augmented with supervised signal mitigates this vanishing gradient issue. Finally, we prove our main result that shows supervised learning with adversarial features can be better than sole supervision (under some mild assumptions). We support our main result on two fronts (i) expected empirical risk and (ii) rate of convergence.