Towards Deep Neural Network Architectures Robust to Adversarial Examples
This addresses a critical security issue for AI systems using deep learning, but the solution is incremental as it builds on existing contractive autoencoder ideas.
The paper tackled the problem of deep neural networks being vulnerable to adversarial examples, which cause high misclassification rates, and found that denoising autoencoders can remove noise but are not fully robust, leading to the proposal of a Deep Contractive Network that improves robustness without significant performance loss.
Recent work has shown deep neural networks (DNNs) to be highly susceptible to well-designed, small perturbations at the input layer, or so-called adversarial examples. Taking images as an example, such distortions are often imperceptible, but can result in 100% mis-classification for a state of the art DNN. We study the structure of adversarial examples and explore network topology, pre-processing and training strategies to improve the robustness of DNNs. We perform various experiments to assess the removability of adversarial examples by corrupting with additional noise and pre-processing with denoising autoencoders (DAEs). We find that DAEs can remove substantial amounts of the adversarial noise. How- ever, when stacking the DAE with the original DNN, the resulting network can again be attacked by new adversarial examples with even smaller distortion. As a solution, we propose Deep Contractive Network, a model with a new end-to-end training procedure that includes a smoothness penalty inspired by the contractive autoencoder (CAE). This increases the network robustness to adversarial examples, without a significant performance penalty.