You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms
This work addresses the challenge of reducing design costs for resource-constrained embedded platforms, offering a more efficient solution compared to incremental improvements in existing NAS methods.
The paper tackles the problem of efficiently designing deep neural networks that meet strict performance constraints, such as runtime latency on autonomous vehicles, by introducing LightNAS, a lightweight hardware-aware differentiable neural architecture search framework that finds suitable architectures through a one-time search, achieving superior results over previous state-of-the-art methods.
Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., \underline{\textit{you only search once}}). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods.