Exploring Adversarial Examples for Efficient Active Learning in Machine Learning Classifiers
This provides a theoretical foundation for using adversarial examples constructively in active learning, potentially benefiting machine learning practitioners seeking efficient training methods.
The paper tackles the lack of theoretical analysis for using training samples near decision boundaries to improve active learning efficiency, by generating adversarial examples to approximate boundaries and proving connections across classifiers, resulting in support for more efficient active learning strategies.
Machine learning researchers have long noticed the phenomenon that the model training process will be more effective and efficient when the training samples are densely sampled around the underlying decision boundary. While this observation has already been widely applied in a range of machine learning security techniques, it lacks theoretical analyses of the correctness of the observation. To address this challenge, we first add particular perturbation to original training examples using adversarial attack methods so that the generated examples could lie approximately on the decision boundary of the ML classifiers. We then investigate the connections between active learning and these particular training examples. Through analyzing various representative classifiers such as k-NN classifiers, kernel methods as well as deep neural networks, we establish a theoretical foundation for the observation. As a result, our theoretical proofs provide support to more efficient active learning methods with the help of adversarial examples, contrary to previous works where adversarial examples are often used as destructive solutions. Experimental results show that the established theoretical foundation will guide better active learning strategies based on adversarial examples.