Regularized Classification-Aware Quantization
This work addresses the need for robust quantization in classification tasks where online relearning is not feasible, representing an incremental improvement over existing methods.
The paper tackles the problem of designing quantization schemes for binary classification tasks by minimizing classification loss rather than reconstruction error, and introduces a method that regularizes the 0-1 loss with reconstruction error, resulting in faster performance proportional to a quadratic term of dataset size compared to previous methods.
Traditionally, quantization is designed to minimize the reconstruction error of a data source. When considering downstream classification tasks, other measures of distortion can be of interest; such as the 0-1 classification loss. Furthermore, it is desirable that the performance of these quantizers not deteriorate once they are deployed into production, as relearning the scheme online is not always possible. In this work, we present a class of algorithms that learn distributed quantization schemes for binary classification tasks. Our method performs well on unseen data, and is faster than previous methods proportional to a quadratic term of the dataset size. It works by regularizing the 0-1 loss with the reconstruction error. We present experiments on synthetic mixture and bivariate Gaussian data and compare training, testing, and generalization errors with a family of benchmark quantization schemes from the literature. Our method is called Regularized Classification-Aware Quantization.