ODG-Q: Robust Quantization via Online Domain Generalization
This work addresses the vulnerability of quantized networks to adversarial attacks, a critical issue for deploying AI models on resource-limited edge hardware, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of making quantized neural networks robust to adversarial attacks by proposing ODG-Q, which frames robust quantization as an online domain generalization problem, achieving significant improvements such as 49.2% average gains under white-box attacks and 21.7% under black-box attacks on CIFAR-10 with training costs similar to natural training.
Quantizing neural networks to low-bitwidth is important for model deployment on resource-limited edge hardware. Although a quantized network has a smaller model size and memory footprint, it is fragile to adversarial attacks. However, few methods study the robustness and training efficiency of quantized networks. To this end, we propose a new method by recasting robust quantization as an online domain generalization problem, termed ODG-Q, which generates diverse adversarial data at a low cost during training. ODG-Q consistently outperforms existing works against various adversarial attacks. For example, on CIFAR-10 dataset, ODG-Q achieves 49.2% average improvements under five common white-box attacks and 21.7% average improvements under five common black-box attacks, with a training cost similar to that of natural training (viz. without adversaries). To our best knowledge, this work is the first work that trains both quantized and binary neural networks on ImageNet that consistently improve robustness under different attacks. We also provide a theoretical insight of ODG-Q that accounts for the bound of model risk on attacked data.