With Shared Microexponents, A Little Shifting Goes a Long Way
This addresses the need for improved quantization methods to enhance computational efficiency in deep learning applications, though it appears incremental as it builds on existing quantization standards.
The paper tackles the problem of efficient narrow-precision formats for deep learning by introducing Block Data Representations (BDR) and shared microexponents (MX), which outperform state-of-the-art quantization approaches in real-world models like generative pretraining and recommendation systems.
This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables comparison of popular quantization standards, and through BDR, new formats based on shared microexponents (MX) are identified, which outperform other state-of-the-art quantization approaches, including narrow-precision floating-point and block floating-point. MX utilizes multiple levels of quantization scaling with ultra-fine scaling factors based on shared microexponents in the hardware. The effectiveness of MX is demonstrated on real-world models including large-scale generative pretraining and inferencing, and production-scale recommendation systems.