HAO: Hardware-aware neural Architecture Optimization for Efficient Inference
This work addresses the problem of efficient DNN inference on FPGAs for applications like image classification, representing an incremental improvement over existing hardware-aware NAS methods.
The paper tackles the challenge of optimizing neural network architectures and hardware accelerators for efficient inference on FPGAs by incorporating integer programming to prune the design space, resulting in quantized networks that achieve 72.5% top-1 accuracy on ImageNet at 50 FPS, which is 60% faster than MnasNet and 135% faster than FBNet with comparable accuracy.
Automatic algorithm-hardware co-design for DNN has shown great success in improving the performance of DNNs on FPGAs. However, this process remains challenging due to the intractable search space of neural network architectures and hardware accelerator implementation. Differing from existing hardware-aware neural architecture search (NAS) algorithms that rely solely on the expensive learning-based approaches, our work incorporates integer programming into the search algorithm to prune the design space. Given a set of hardware resource constraints, our integer programming formulation directly outputs the optimal accelerator configuration for mapping a DNN subgraph that minimizes latency. We use an accuracy predictor for different DNN subgraphs with different quantization schemes and generate accuracy-latency pareto frontiers. With low computational cost, our algorithm can generate quantized networks that achieve state-of-the-art accuracy and hardware performance on Xilinx Zynq (ZU3EG) FPGA for image classification on ImageNet dataset. The solution searched by our algorithm achieves 72.5% top-1 accuracy on ImageNet at framerate 50, which is 60% faster than MnasNet and 135% faster than FBNet with comparable accuracy.