A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction
This work addresses the computational bottleneck in NAS for computer vision researchers by enabling more efficient performance prediction, though it is incremental as it builds on existing GNN and autoencoder methods.
The paper tackles the problem of predicting neural architecture performance for Neural Architecture Search (NAS) by proposing a graph encoder based on Graph Neural Networks (GNNs) and a variational-sequential graph autoencoder (VS-GAE), demonstrating effectiveness in both seen and unseen architecture types and showing that the learned embedding space increases stability in accuracy prediction.
In computer vision research, the process of automating architecture engineering, Neural Architecture Search (NAS), has gained substantial interest. In the past, NAS was hardly accessible to researchers without access to large-scale compute systems, due to very long compute times for the recurrent search and evaluation of new candidate architectures. The NAS-Bench-101 dataset facilitates a paradigm change towards classical methods such as supervised learning to evaluate neural architectures. In this paper, we propose a graph encoder built upon Graph Neural Networks (GNN). We demonstrate the effectiveness of the proposed encoder on NAS performance prediction for seen architecture types as well an unseen ones (i.e., zero shot prediction). We also provide a new variational-sequential graph autoencoder (VS-GAE) based on the proposed graph encoder. The VS-GAE is specialized on encoding and decoding graphs of varying length utilizing GNNs. Experiments on different sampling methods show that the embedding space learned by our VS-GAE increases the stability on the accuracy prediction task.