Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
This addresses the need for efficient compression techniques in deep learning applications, offering a unified approach for tasks like image and model compression, though it appears incremental as it builds on existing quantization methods.
The paper tackles the problem of learning compressible representations in deep architectures by introducing a soft-to-hard vector quantization method that anneals from continuous relaxations to discrete quantization and entropy, achieving results competitive with state-of-the-art in image compression and neural network compression.
We present a new approach to learn compressible representations in deep architectures with an end-to-end training strategy. Our method is based on a soft (continuous) relaxation of quantization and entropy, which we anneal to their discrete counterparts throughout training. We showcase this method for two challenging applications: Image compression and neural network compression. While these tasks have typically been approached with different methods, our soft-to-hard quantization approach gives results competitive with the state-of-the-art for both.