QONNX: Representing Arbitrary-Precision Quantized Neural Networks
This work addresses the need for standardized representation of quantized neural networks to facilitate deployment across various hardware platforms, though it is incremental as it builds upon existing ONNX formats.
The authors introduced QONNX, an extension to the ONNX format for representing arbitrary-precision quantized neural networks, enabling support for low-precision quantization and targeting diverse platforms, with utilities and a model zoo provided for toolchains like FINN and hls4ml.
We present extensions to the Open Neural Network Exchange (ONNX) intermediate representation format to represent arbitrary-precision quantized neural networks. We first introduce support for low precision quantization in existing ONNX-based quantization formats by leveraging integer clipping, resulting in two new backward-compatible variants: the quantized operator format with clipping and quantize-clip-dequantize (QCDQ) format. We then introduce a novel higher-level ONNX format called quantized ONNX (QONNX) that introduces three new operators -- Quant, BipolarQuant, and Trunc -- in order to represent uniform quantization. By keeping the QONNX IR high-level and flexible, we enable targeting a wider variety of platforms. We also present utilities for working with QONNX, as well as examples of its usage in the FINN and hls4ml toolchains. Finally, we introduce the QONNX model zoo to share low-precision quantized neural networks.