DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression
This addresses the problem of reducing storage and transmission costs for deep neural networks, particularly for resource-constrained applications, with a domain-specific incremental improvement.
The paper tackles deep neural network compression by introducing DeepCABAC, a context-adaptive binary arithmetic coder that quantizes weights to minimize rate-distortion while preserving accuracy, achieving a compression ratio of x63.6 for VGG16 on ImageNet with no loss, reducing the model to 8.7MB.
We present DeepCABAC, a novel context-adaptive binary arithmetic coder for compressing deep neural networks. It quantizes each weight parameter by minimizing a weighted rate-distortion function, which implicitly takes the impact of quantization on to the accuracy of the network into account. Subsequently, it compresses the quantized values into a bitstream representation with minimal redundancies. We show that DeepCABAC is able to reach very high compression ratios across a wide set of different network architectures and datasets. For instance, we are able to compress by x63.6 the VGG16 ImageNet model with no loss of accuracy, thus being able to represent the entire network with merely 8.7MB.