Trends in Neural Architecture Search: Towards the Acceleration of Search
This is an incremental survey paper that organizes existing knowledge for researchers in deep learning and NAS.
The paper surveys and classifies major trends in neural architecture search (NAS), including neuro-evolutionary algorithms, reinforcement learning based methods, and one-shot approaches, and compares them while discussing future directions.
In modern deep learning research, finding optimal (or near optimal) neural network models is one of major research directions and it is widely studied in many applications. In this paper, the main research trends of neural architecture search (NAS) are classified as neuro-evolutionary algorithms, reinforcement learning based algorithms, and one-shot architecture search approaches. Furthermore, each research trend is introduced and finally all the major three trends are compared. Lastly, the future research directions of NAS research trends are discussed.