Prior-Guided One-shot Neural Architecture Search
This work addresses a specific bottleneck in neural architecture search for researchers and practitioners, offering an incremental improvement to enhance ranking correlation in supernet-based methods.
The paper tackles the problem of poor ranking consistency between stand-alone architectures and shared-weight supernets in neural architecture search by introducing Prior-Guided One-shot NAS (PGONAS), which uses a balanced sampling strategy and ranking correlation loss to improve supernet training, achieving 3rd place in the CVPR2022 lightweight NAS challenge supernet track.
Neural architecture search methods seek optimal candidates with efficient weight-sharing supernet training. However, recent studies indicate poor ranking consistency about the performance between stand-alone architectures and shared-weight networks. In this paper, we present Prior-Guided One-shot NAS (PGONAS) to strengthen the ranking correlation of supernets. Specifically, we first explore the effect of activation functions and propose a balanced sampling strategy based on the Sandwich Rule to alleviate weight coupling in the supernet. Then, FLOPs and Zen-Score are adopted to guide the training of supernet with ranking correlation loss. Our PGONAS ranks 3rd place in the supernet Track Track of CVPR2022 Second lightweight NAS challenge. Code is available in https://github.com/pprp/CVPR2022-NAS?competition-Track1-3th-solution.