The Fourth State: Signed-Zero Ternary for Stable LLM Quantization (and More)
This addresses the challenge of reducing compute requirements while maintaining performance for large language models and other AI applications, though it appears incremental as it builds on existing quantization techniques.
The paper tackles the problem of quantization in machine learning by introducing Signed-Zero Ternary (SZT), a 2-bit quantization method that offers gradient information without forward-path penalties, potentially improving information density compared to non-quantized alternatives.
Quantization is usually regarded as a means to trade quality of performance for reduced compute requirements, i.e., as a suboptimal approximation. However, if examined in terms of a fixed overall resource budget, a very different perspective arises. We introduce Signed-Zero Ternary (SZT), a 2-bit quantization that deterministically provides gradient information with no forward-path penalty. Our analysis provides evidence that it may improve information density compared to non-quantized alternatives.