Multi-trial Neural Architecture Search with Lottery Tickets
This work addresses the problem of limited performance in NAS for image recognition by enabling more efficient and effective architecture search, though it appears incremental as it builds on existing NAS and pruning techniques.
The paper tackles the limitation of restricted search spaces in neural architecture search (NAS) by proposing MobileNet3-MT, a new search space with reduced human prior knowledge, and MENAS, a multi-trial evolution-based method that accelerates search using lottery tickets and pruning. It achieves state-of-the-art performance on ImageNet-1K, CIFAR-10, and CIFAR-100 datasets.
Neural architecture search (NAS) has brought significant progress in recent image recognition tasks. Most existing NAS methods apply restricted search spaces, which limits the upper-bound performance of searched models. To address this issue, we propose a new search space named MobileNet3-MT. By reducing human-prior knowledge in omni dimensions of networks, MobileNet3-MT accommodates more potential candidates. For searching in this challenging search space, we present an efficient Multi-trial Evolution-based NAS method termed MENAS. Specifically, we accelerate the evolutionary search process by gradually pruning models in the population. Each model is trained with an early stop and replaced by its Lottery Tickets (the explored optimal pruned network).In this way, the full training pipeline of cumbersome networks is prevented and more efficient networks are automatically generated. Extensive experimental results on ImageNet-1K, CIFAR-10, and CIFAR-100 demonstrate that MENAS achieves state-of-the-art performance.