End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs
This work addresses the need for accessible, efficient hardware implementations of neural networks for real-time machine learning in scientific and industrial settings, representing an incremental improvement by combining existing tools into a unified workflow.
The authors tackled the problem of efficiently implementing neural networks on FPGA and ASIC hardware for real-time applications by developing an end-to-end workflow that integrates Hessian-aware quantization, QONNX, and hls4ml, demonstrating it in a particle physics application at CERN's LHC with a 40 MHz collision rate.
We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) hardware. Our approach leverages Hessian-aware quantization (HAWQ) of NNs, the Quantized Open Neural Network Exchange (QONNX) intermediate representation, and the hls4ml tool flow for transpiling NNs into FPGA and ASIC firmware. This makes efficient NN implementations in hardware accessible to nonexperts, in a single open-sourced workflow that can be deployed for real-time machine learning applications in a wide range of scientific and industrial settings. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40 MHz collision rate of the CERN Large Hadron Collider (LHC). Given the high collision rate, all data processing must be implemented on custom ASIC and FPGA hardware within a strict area and latency. Based on these constraints, we implement an optimized mixed-precision NN classifier for high-momentum particle jets in simulated LHC proton-proton collisions.