Finding Non-Uniform Quantization Schemes using Multi-Task Gaussian Processes
This work addresses memory efficiency in deep learning for computer vision applications, offering a method to reduce model size with little performance drop, though it appears incremental as it builds on existing quantization and neural architecture search techniques.
The paper tackles neural network quantization by framing it as a hyperparameter search to find non-uniform bit distributions across CNN layers, using Multi-Task Gaussian Processes to explore configurations efficiently, and shows minimal accuracy loss with significant memory savings on datasets like CIFAR10 and ImageNet.
We propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the problem to multiple tasks, each corresponding to different number of training epochs, and explore the space by sampling those configurations that yield maximum information. We then show that with significantly lower precision in the last layers we achieve a minimal loss of accuracy with appreciable memory savings. We test our findings on the CIFAR10 and ImageNet datasets using the VGG, ResNet and GoogLeNet architectures.