Analyzing Compression Techniques for Computer Vision
This work addresses compression for practical computer vision applications, but it is incremental as it tests existing methods on standard datasets.
The paper explored combining knowledge distillation, pruning, and quantization for compressing deep networks in computer vision, analyzing their efficacy on MNIST and CIFAR-10 datasets and presenting results with observations.
Compressing deep networks is highly desirable for practical use-cases in computer vision applications. Several techniques have been explored in the literature, and research has been done in finding efficient strategies for combining them. For this project, we aimed to explore three different basic compression techniques - knowledge distillation, pruning, and quantization for small-scale recognition tasks. Along with the basic methods, we also test the efficacy of combining them in a sequential manner. We analyze them using MNIST and CIFAR-10 datasets and present the results along with few observations inferred from them.