A Semi-Supervised Assessor of Neural Architectures
This addresses the efficiency bottleneck in neural architecture search for researchers and practitioners, though it is incremental as it builds on existing performance prediction methods.
The paper tackles the problem of reducing the number of fully trained neural architectures needed for neural architecture search by proposing a semi-supervised assessor that uses an auto-encoder and graph convolutional network to predict performance, achieving a significant reduction in required architectures on the NAS-Benchmark-101 dataset.
Neural architecture search (NAS) aims to automatically design deep neural networks of satisfactory performance. Wherein, architecture performance predictor is critical to efficiently value an intermediate neural architecture. But for the training of this predictor, a number of neural architectures and their corresponding real performance often have to be collected. In contrast with classical performance predictor optimized in a fully supervised way, this paper suggests a semi-supervised assessor of neural architectures. We employ an auto-encoder to discover meaningful representations of neural architectures. Taking each neural architecture as an individual instance in the search space, we construct a graph to capture their intrinsic similarities, where both labeled and unlabeled architectures are involved. A graph convolutional neural network is introduced to predict the performance of architectures based on the learned representations and their relation modeled by the graph. Extensive experimental results on the NAS-Benchmark-101 dataset demonstrated that our method is able to make a significant reduction on the required fully trained architectures for finding efficient architectures.