Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
This addresses the efficiency bottleneck in NAS for deep image recognition, enabling faster architecture search with competitive performance, though it is incremental as it builds on existing NAS frameworks.
The paper tackles the high computational cost of building accuracy predictors in Neural Architecture Search (NAS) by proposing Zen-Score, a zero-shot index that correlates with model accuracy and requires only a few forward inferences without training, leading to Zen-NAS which achieves state-of-the-art accuracy on ImageNet within less than half a GPU day.
Accuracy predictor is a key component in Neural Architecture Search (NAS) for ranking architectures. Building a high-quality accuracy predictor usually costs enormous computation. To address this issue, instead of using an accuracy predictor, we propose a novel zero-shot index dubbed Zen-Score to rank the architectures. The Zen-Score represents the network expressivity and positively correlates with the model accuracy. The calculation of Zen-Score only takes a few forward inferences through a randomly initialized network, without training network parameters. Built upon the Zen-Score, we further propose a new NAS algorithm, termed as Zen-NAS, by maximizing the Zen-Score of the target network under given inference budgets. Within less than half GPU day, Zen-NAS is able to directly search high performance architectures in a data-free style. Comparing with previous NAS methods, the proposed Zen-NAS is magnitude times faster on multiple server-side and mobile-side GPU platforms with state-of-the-art accuracy on ImageNet. Our source code and pre-trained models are released on https://github.com/idstcv/ZenNAS.