Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models
This addresses the problem of deploying robust and efficient models on resource-constrained devices like mobile and edge systems, representing an incremental improvement over existing adversarial pruning techniques.
The paper tackles the vulnerability of compressed deep learning models to adversarial attacks by introducing Efficient Ensemble Defense (EED), which diversifies compression via pruning and enhances ensemble diversity, achieving state-of-the-art robustness and up to 1.86 times faster inference on CIFAR-10 and SVHN datasets.
Deep learning-based computer vision systems adopt complex and large architectures to improve performance, yet they face challenges in deployment on resource-constrained mobile and edge devices. To address this issue, model compression techniques such as pruning, quantization, and matrix factorization have been proposed; however, these compressed models are often highly vulnerable to adversarial attacks. We introduce the \textbf{Efficient Ensemble Defense (EED)} technique, which diversifies the compression of a single base model based on different pruning importance scores and enhances ensemble diversity to achieve high adversarial robustness and resource efficiency. EED dynamically determines the number of necessary sub-models during the inference stage, minimizing unnecessary computations while maintaining high robustness. On the CIFAR-10 and SVHN datasets, EED demonstrated state-of-the-art robustness performance compared to existing adversarial pruning techniques, along with an inference speed improvement of up to 1.86 times. This proves that EED is a powerful defense solution in resource-constrained environments.