A Survey on Computationally Efficient Neural Architecture Search
It tackles the problem of making NAS more accessible and practical for users without deep expertise by reducing computational burdens, but it is incremental as it synthesizes existing work rather than introducing new methods.
This survey addresses the computational inefficiency of neural architecture search (NAS) by providing a comprehensive overview of computationally efficient NAS methods, categorizing them into proxy-based and surrogate-assisted approaches and comparing their performances and complexities.
Neural architecture search (NAS) has become increasingly popular in the deep learning community recently, mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks (DNNs). However, NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS, and training DNNs is computationally intensive. To solve this major limitation of NAS, improving the computational efficiency is essential in the design of NAS. However, a systematic overview of computationally efficient NAS (CE-NAS) methods still lacks. To fill this gap, we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods, together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities. The remaining challenges and open research questions are also discussed, and promising research topics in this emerging field are suggested.