Core-set Sampling for Efficient Neural Architecture Search
This addresses the computational burden in NAS for researchers and practitioners, offering a significant speed-up while maintaining performance, though it is incremental as it builds on existing NAS frameworks.
The paper tackles the problem of reducing the large search time in neural architecture search (NAS) by using core-set sampling to search architectures on summarized data distribution, achieving an 8.8x reduction in computational time from 30.8 hours to 3.5 hours without sacrificing accuracy.
Neural architecture search (NAS), an important branch of automatic machine learning, has become an effective approach to automate the design of deep learning models. However, the major issue in NAS is how to reduce the large search time imposed by the heavy computational burden. While most recent approaches focus on pruning redundant sets or developing new search methodologies, this paper attempts to formulate the problem based on the data curation manner. Our key strategy is to search the architecture using summarized data distribution, i.e., core-set. Typically, many NAS algorithms separate searching and training stages, and the proposed core-set methodology is only used in search stage, thus their performance degradation can be minimized. In our experiments, we were able to save overall computational time from 30.8 hours to 3.5 hours, 8.8x reduction, on a single RTX 3090 GPU without sacrificing accuracy.