Pre-Quantized Deep Learning Models Codified in ONNX to Enable Hardware/Software Co-Design
This work addresses the challenge of hardware/software co-design for deep learning practitioners, but it appears incremental as it builds on existing ONNX standards without introducing a new paradigm.
The paper tackles the problem of integrating quantization into deep learning models by proposing a methodology to separate quantization from hardware-specific compilation using a pre-quantized ONNX format, enabling independent development and hardware/software co-design.
This paper presents a methodology to separate the quantization process from the hardware-specific model compilation stage via a pre-quantized deep learning model description in standard ONNX format. Separating the quantization process from the model compilation stage enables independent development. The methodology is expressive to convey hardware-specific operations and to embed key quantization parameters into a ONNX model which enables hardware/software co-design. Detailed examples are given for both MLP and CNN based networks, which can be extended to other networks in a straightforward fashion.