Efficient Neural Architecture Search with Performance Prediction
This work addresses the efficiency problem in NAS for machine learning practitioners, offering an incremental improvement over existing methods.
The paper tackles the high computational cost of neural architecture search (NAS) by proposing an end-to-end offline performance predictor to accelerate architecture evaluation, reducing the need for full training from scratch.
Neural networks are powerful models that have a remarkable ability to extract patterns that are too complex to be noticed by humans or other machine learning models. Neural networks are the first class of models that can train end-to-end systems with large learning capacities. However, we still have the difficult challenge of designing the neural network, which requires human experience and a long process of trial and error. As a solution, we can use a neural architecture search to find the best network architecture for the task at hand. Existing NAS algorithms generally evaluate the fitness of a new architecture by fully training from scratch, resulting in the prohibitive computational cost, even if operated on high-performance computers. In this paper, an end-to-end offline performance predictor is proposed to accelerate the evaluation of sampled architectures. Index Terms- Learning Curve Prediction, Neural Architecture Search, Reinforcement Learning.