DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning
This work addresses efficiency and generality issues in NAS, making it practical for on-device models, but it is incremental as it builds on existing NAS methods with improvements in speed and performance.
The paper tackles the problem of high computational complexity and low generality in Neural Architecture Search (NAS) by proposing DDPNAS, an efficient framework using dynamic distribution pruning, which achieves top-1 accuracies of 97.56% on CIFAR-10 and 77.2% on ImageNet with a search time of only 1.8 GPU hours.
Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints. The architectures searched by our method achieve remarkable top-1 accuracies, 97.56 and 77.2 on CIFAR-10 and ImageNet (mobile settings), respectively, with the fastest search process, i.e., only 1.8 GPU hours on a Tesla V100. Codes for searching and network generation are available at: https://openi.pcl.ac.cn/PCL AutoML/XNAS.