Binary and Ternary Quantization Can Enhance Feature Discrimination
This addresses the problem of optimizing computational efficiency in machine learning for practitioners, though it is incremental as it builds on existing quantization research.
The paper tackles the problem of how quantization affects classification performance by challenging the assumption that larger quantization errors reduce accuracy, showing that binary and ternary quantization can enhance feature discrimination and achieve comparable or superior accuracy to full-precision data in experiments.
Quantization is widely applied in machine learning to reduce computational and storage costs for both data and models. Considering that classification tasks are fundamental to the field, it is crucial to investigate how quantization impacts classification performance. Traditional research has focused on quantization errors, assuming that larger errors generally lead to lower classification accuracy. However, this assumption lacks a solid theoretical foundation and often contradicts empirical observations. For example, despite introducing significant errors, $\{0,1\}$-binary and $\{0, \pm1\}$-ternary quantized data have sometimes achieved classification accuracy comparable or even superior to full-precision data. To reasonably explain this phenomenon, a more accurate evaluation of classification performance is required. To achieve this, we propose a direct analysis of the feature discrimination of quantized data, instead of focusing on quantization errors. Our analysis reveals that both binary and ternary quantization can potentially enhance, rather than degrade, the feature discrimination of the original data. This finding is supported by classification experiments conducted on both synthetic and real data.