REx: Data-Free Residual Quantization Error Expansion
This addresses the need for flexible, privacy-preserving quantization to adapt to diverse hardware devices, offering a solution for efficient deployment in computer vision and NLP.
The paper tackles the problem of high inference cost in deep neural networks by proposing REx, a data-free quantization method that achieves superior performance across various benchmarks, architectures, and bit-widths, including int8 to ternary quantization.
Deep neural networks (DNNs) are ubiquitous in computer vision and natural language processing, but suffer from high inference cost. This problem can be addressed by quantization, which consists in converting floating point operations into a lower bit-width format. With the growing concerns on privacy rights, we focus our efforts on data-free methods. However, such techniques suffer from their lack of adaptability to the target devices, as a hardware typically only support specific bit widths. Thus, to adapt to a variety of devices, a quantization method shall be flexible enough to find good accuracy v.s. speed trade-offs for every bit width and target device. To achieve this, we propose REx, a quantization method that leverages residual error expansion, along with group sparsity and an ensemble approximation for better parallelization. REx is backed off by strong theoretical guarantees and achieves superior performance on every benchmarked application (from vision to NLP tasks), architecture (ConvNets, transformers) and bit-width (from int8 to ternary quantization).