Frame Quantization of Neural Networks
This work addresses efficient neural network deployment for resource-constrained applications, but it is incremental as it builds on existing quantization and frame theory methods.
The paper tackles the problem of neural network quantization by introducing a post-training algorithm based on frame theory and Sigma-Delta quantization, resulting in an error bound dependent on step size and frame elements and demonstrating improved accuracy through frame redundancy.
We present a post-training quantization algorithm with error estimates relying on ideas originating from frame theory. Specifically, we use first-order Sigma-Delta ($ΣΔ$) quantization for finite unit-norm tight frames to quantize weight matrices and biases in a neural network. In our scenario, we derive an error bound between the original neural network and the quantized neural network in terms of step size and the number of frame elements. We also demonstrate how to leverage the redundancy of frames to achieve a quantized neural network with higher accuracy.