NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search
This work addresses the need for sample-efficient and versatile neural architecture search for researchers and practitioners handling diverse datasets, though it is incremental as it builds on prior NAS methods.
The paper tackles the problem of designing efficient neural architectures under multiple competing objectives by proposing NSGANetV2, a surrogate-assisted NAS algorithm that matches or outperforms existing models on standard benchmarks like C10, C100, and ImageNet while being orders of magnitude more sample efficient.
In this paper, we propose an efficient NAS algorithm for generating task-specific models that are competitive under multiple competing objectives. It comprises of two surrogates, one at the architecture level to improve sample efficiency and one at the weights level, through a supernet, to improve gradient descent training efficiency. On standard benchmark datasets (C10, C100, ImageNet), the resulting models, dubbed NSGANetV2, either match or outperform models from existing approaches with the search being orders of magnitude more sample efficient. Furthermore, we demonstrate the effectiveness and versatility of the proposed method on six diverse non-standard datasets, e.g. STL-10, Flowers102, Oxford Pets, FGVC Aircrafts etc. In all cases, NSGANetV2s improve the state-of-the-art (under mobile setting), suggesting that NAS can be a viable alternative to conventional transfer learning approaches in handling diverse scenarios such as small-scale or fine-grained datasets. Code is available at https://github.com/mikelzc1990/nsganetv2