Hardware-aware Neural Architecture Search of Early Exiting Networks on Edge Accelerators
This work addresses efficiency problems for edge computing applications, but it is incremental as it builds on existing early exiting and NAS methods.
The paper tackles the challenge of deploying deep learning models on edge devices with computational constraints by proposing a hardware-aware Neural Architecture Search framework to optimize early exiting networks, achieving over a 50% reduction in computational costs on CIFAR-10 compared to static networks.
Advancements in high-performance computing and cloud technologies have enabled the development of increasingly sophisticated Deep Learning (DL) models. However, the growing demand for embedded intelligence at the edge imposes stringent computational and energy constraints, challenging the deployment of these large-scale models. Early Exiting Neural Networks (EENN) have emerged as a promising solution, allowing dynamic termination of inference based on input complexity to enhance efficiency. Despite their potential, EENN performance is highly influenced by the heterogeneity of edge accelerators and the constraints imposed by quantization, affecting accuracy, energy efficiency, and latency. Yet, research on the automatic optimization of EENN design for edge hardware remains limited. To bridge this gap, we propose a hardware-aware Neural Architecture Search (NAS) framework that systematically integrates the effects of quantization and hardware resource allocation to optimize the placement of early exit points within a network backbone. Experimental results on the CIFAR-10 dataset demonstrate that our NAS framework can discover architectures that achieve over a 50\% reduction in computational costs compared to conventional static networks, making them more suitable for deployment in resource-constrained edge environments.