U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture Search
This work addresses the challenge of inefficient resource utilization in NAS for inference accelerators like TPUs, which is an incremental improvement over existing hardware-aware methods.
The paper tackled the problem of optimizing resource utilization in hardware-aware neural architecture search (NAS) for DNN inference, achieving a 2.8-4x speedup compared to prior methods while maintaining or improving accuracy on CIFAR-10 and ImageNet-100 datasets.
Optimizing resource utilization in target platforms is key to achieving high performance during DNN inference. While optimizations have been proposed for inference latency, memory footprint, and energy consumption, prior hardware-aware neural architecture search (NAS) methods have omitted resource utilization, preventing DNNs to take full advantage of the target inference platforms. Modeling resource utilization efficiently and accurately is challenging, especially for widely-used array-based inference accelerators such as Google TPU. In this work, we propose a novel hardware-aware NAS framework that does not only optimize for task accuracy and inference latency, but also for resource utilization. We also propose and validate a new computational model for resource utilization in inference accelerators. By using the proposed NAS framework and the proposed resource utilization model, we achieve 2.8 - 4x speedup for DNN inference compared to prior hardware-aware NAS methods while attaining similar or improved accuracy in image classification on CIFAR-10 and Imagenet-100 datasets.