Learning with Multiplicative Perturbations
This work addresses the need for more efficient and interpretable adversarial training methods in machine learning, though it appears incremental as it builds directly on existing AT and VAT techniques.
The paper tackles the problem of adversarial training for deep neural networks by proposing new algorithms (xAT and xVAT) that use multiplicative perturbations instead of additive ones, resulting in matching or outperforming state-of-the-art classification accuracies across benchmarks while being about 30% faster.
Adversarial Training (AT) and Virtual Adversarial Training (VAT) are the regularization techniques that train Deep Neural Networks (DNNs) with adversarial examples generated by adding small but worst-case perturbations to input examples. In this paper, we propose xAT and xVAT, new adversarial training algorithms, that generate \textbf{multiplicative} perturbations to input examples for robust training of DNNs. Such perturbations are much more perceptible and interpretable than their \textbf{additive} counterparts exploited by AT and VAT. Furthermore, the multiplicative perturbations can be generated transductively or inductively while the standard AT and VAT only support a transductive implementation. We conduct a series of experiments that analyze the behavior of the multiplicative perturbations and demonstrate that xAT and xVAT match or outperform state-of-the-art classification accuracies across multiple established benchmarks while being about 30\% faster than their additive counterparts. Furthermore, the resulting DNNs also demonstrate distinct weight distributions.