Defensive Tensorization
This work addresses adversarial robustness for neural networks, offering a versatile defense that can be integrated with various architectures, but it appears incremental as it builds on existing tensorization and dropout techniques.
The paper tackles adversarial attacks on neural networks by introducing defensive tensorization, which factorizes network layers into tensors and applies tensor dropout in the latent subspace to avoid sparsity or perturbations, resulting in improved performance on image and audio classification benchmarks compared to prior works.
We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network. The layers of a network are first expressed as factorized tensor layers. Tensor dropout is then applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization.Our approach can be readily integrated with any arbitrary neural architecture and combined with techniques like adversarial training. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We validate the versatility of our approach across domains and low-precision architectures by considering an audio classification task and binary networks. In all cases, we demonstrate improved performance compared to prior works.