A scalable constructive algorithm for the optimization of neural network architectures
This addresses the challenge of efficient hyperparameter optimization for neural networks, offering a scalable solution that could benefit researchers and practitioners in machine learning, though it appears incremental as it builds on existing search methods.
The authors tackled the problem of optimizing neural network architectures by proposing a scalable constructive algorithm that determines a minimal-layer network with performance at least as good as other hyperparameter search methods. Numerical results on benchmark datasets show it outperforms state-of-the-art algorithms in predictive performance and time-to-solution.
We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified by other hyperparameter search algorithms in terms of accuracy and computational cost. Numerical results performed on benchmark datasets show that, for these datasets, our method outperforms state-of-the-art hyperparameter optimization algorithms in terms of attainable predictive performance by the selected neural network architecture, and time-to-solution for the hyperparameter optimization to complete.