Reducing Neural Architecture Search Spaces with Training-Free Statistics and Computational Graph Clustering
This work addresses computational efficiency for researchers and practitioners using NAS, though it is incremental as it builds on existing NAS methods.
The paper tackles the high computational cost of neural architecture search (NAS) by introducing C-BRED, a method that reduces search space size through clustering and proxy statistics, achieving a 70% average accuracy on CIFAR-100 compared to 64% for the full space.
The computational demands of neural architecture search (NAS) algorithms are usually directly proportional to the size of their target search spaces. Thus, limiting the search to high-quality subsets can greatly reduce the computational load of NAS algorithms. In this paper, we present Clustering-Based REDuction (C-BRED), a new technique to reduce the size of NAS search spaces. C-BRED reduces a NAS space by clustering the computational graphs associated with its architectures and selecting the most promising cluster using proxy statistics correlated with network accuracy. When considering the NAS-Bench-201 (NB201) data set and the CIFAR-100 task, C-BRED selects a subset with 70% average accuracy instead of the whole space's 64% average accuracy.