Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation
This work addresses the need for better lightweight architectures in neural architecture search, which is an incremental improvement for efficient model design in computer vision.
The paper tackles the problem of designing lightweight neural architectures efficiently by proposing MoARR, a multi-objective neural architecture search algorithm that increases architecture diversity and reduces search space, achieving a model with 1.9% error rate and 2.3M parameters on CIFAR-10 in 6 GPU hours and transferring it to ImageNet with 76.0% top-1 accuracy and 4.9M parameters.
The search space of neural architecture search (NAS) for convolutional neural network (CNN) is huge. To reduce searching cost, most NAS algorithms use fixed outer network level structure, and search the repeatable cell structure only. Such kind of fixed architecture performs well when enough cells and channels are used. However, when the architecture becomes more lightweight, the performance decreases significantly. To obtain better lightweight architectures, more flexible and diversified neural architectures are in demand, and more efficient methods should be designed for larger search space. Motivated by this, we propose MoARR algorithm, which utilizes the existing research results and historical information to quickly find architectures that are both lightweight and accurate. We use the discovered high-performance cells to construct network architectures. This method increases the network architecture diversity while also reduces the search space of cell structure design. In addition, we designs a novel multi-objective method to effectively analyze the historical evaluation information, so as to efficiently search for the Pareto optimal architectures with high accuracy and small parameter number. Experimental results show that our MoARR can achieve a powerful and lightweight model (with 1.9% error rate and 2.3M parameters) on CIFAR-10 in 6 GPU hours, which is better than the state-of-the-arts. The explored architecture is transferable to ImageNet and achieves 76.0% top-1 accuracy with 4.9M parameters.