Understanding and Improving Virtual Adversarial Training
This work addresses the problem of enhancing semi-supervised learning efficiency for researchers and practitioners, though it is incremental as it builds on existing VAT and Bad GAN methods.
The paper tackles the challenge of understanding and improving Virtual Adversarial Training (VAT) in semi-supervised learning by providing a fundamental explanation for its effectiveness and proposing a new technique that integrates Bad GAN ideas without additional generative architectures, achieving competitive performance with fewer computations on benchmark image datasets.
In semi-supervised learning, virtual adversarial training (VAT) approach is one of the most attractive method due to its intuitional simplicity and powerful performances. VAT finds a classifier which is robust to data perturbation toward the adversarial direction. In this study, we provide a fundamental explanation why VAT works well in semi-supervised learning case and propose new techniques which are simple but powerful to improve the VAT method. Especially we employ the idea of Bad GAN approach, which utilizes bad samples distributed on complement of the support of the input data, without any additional deep generative architectures. We generate bad samples of high-quality by use of the adversarial training used in VAT and also give theoretical explanations why the adversarial training is good at both generating bad samples. An advantage of our proposed method is to achieve the competitive performances compared with other recent studies with much fewer computations. We demonstrate advantages our method by various experiments with well known benchmark image datasets.