FTSO: Effective NAS via First Topology Second Operator
This addresses the efficiency problem in NAS for machine learning researchers, offering a significant speed-up with improved accuracy, though it is incremental as it builds on existing one-shot NAS methods.
The paper tackles the high computational cost of neural architecture search (NAS) by proposing FTSO, which divides the search into topology and operator steps, reducing search time from days to 0.68 seconds and improving accuracy, achieving 76.4% on ImageNet (1.5% higher than SOTA) and 97.77% on CIFAR10 (0.27% higher than SOTA).
Existing one-shot neural architecture search (NAS) methods have to conduct a search over a giant super-net, which leads to the huge computational cost. To reduce such cost, in this paper, we propose a method, called FTSO, to divide the whole architecture search into two sub-steps. Specifically, in the first step, we only search for the topology, and in the second step, we search for the operators. FTSO not only reduces NAS's search time from days to 0.68 seconds, but also significantly improves the found architecture's accuracy. Our extensive experiments on ImageNet show that within 18 seconds, FTSO can achieve a 76.4% testing accuracy, 1.5% higher than the SOTA, PC-DARTS. In addition, FTSO can reach a 97.77% testing accuracy, 0.27% higher than the SOTA, with nearly 100% (99.8%) search time saved, when searching on CIFAR10.