Confounding Tradeoffs for Neural Network Quantization
This addresses the problem of inconsistent evaluation in neural network quantization for researchers and engineers, though it is incremental as it builds on existing quantization techniques.
The paper identifies confounding tradeoffs in neural network quantization that can impact accuracy more than the quantization methods themselves, and proposes 'quantization cards' to report these design choices for better comparison and hardware applicability.
Many neural network quantization techniques have been developed to decrease the computational and memory footprint of deep learning. However, these methods are evaluated subject to confounding tradeoffs that may affect inference acceleration or resource complexity in exchange for higher accuracy. In this work, we articulate a variety of tradeoffs whose impact is often overlooked and empirically analyze their impact on uniform and mixed-precision post-training quantization, finding that these confounding tradeoffs may have a larger impact on quantized network accuracy than the actual quantization methods themselves. Because these tradeoffs constrain the attainable hardware acceleration for different use-cases, we encourage researchers to explicitly report these design choices through the structure of "quantization cards." We expect quantization cards to help researchers compare methods more effectively and engineers determine the applicability of quantization techniques for their hardware.