SGAS: Sequential Greedy Architecture Search
This addresses a common problem in automated architecture design for deep learning, offering a more reliable and efficient approach for researchers and practitioners.
The paper tackles the issue of neural architecture search (NAS) methods failing to generalize from validation to final evaluation by introducing SGAS, an efficient method that uses sequential greedy selection and pruning, achieving state-of-the-art results in tasks like image classification and node classification with minimal computational cost.
Architecture design has become a crucial component of successful deep learning. Recent progress in automatic neural architecture search (NAS) shows a lot of promise. However, discovered architectures often fail to generalize in the final evaluation. Architectures with a higher validation accuracy during the search phase may perform worse in the evaluation. Aiming to alleviate this common issue, we introduce sequential greedy architecture search (SGAS), an efficient method for neural architecture search. By dividing the search procedure into sub-problems, SGAS chooses and prunes candidate operations in a greedy fashion. We apply SGAS to search architectures for Convolutional Neural Networks (CNN) and Graph Convolutional Networks (GCN). Extensive experiments show that SGAS is able to find state-of-the-art architectures for tasks such as image classification, point cloud classification and node classification in protein-protein interaction graphs with minimal computational cost. Please visit https://www.deepgcns.org/auto/sgas for more information about SGAS.