Automated Search for Configurations of Deep Neural Network Architectures
This addresses the challenge of automating DNN design for machine learning practitioners, though it appears incremental as it builds on existing configuration and search methods.
The paper tackles the problem of manual configuration and tuning of deep neural networks by proposing an end-to-end framework for automated search of DNN architectures, demonstrating that it identifies high-performing models and outperforms state-of-the-art handcrafted architectures on image classification tasks like MNIST and CIFAR-10.
Deep Neural Networks (DNNs) are intensively used to solve a wide variety of complex problems. Although powerful, such systems require manual configuration and tuning. To this end, we view DNNs as configurable systems and propose an end-to-end framework that allows the configuration, evaluation and automated search for DNN architectures. Therefore, our contribution is threefold. First, we model the variability of DNN architectures with a Feature Model (FM) that generalizes over existing architectures. Each valid configuration of the FM corresponds to a valid DNN model that can be built and trained. Second, we implement, on top of Tensorflow, an automated procedure to deploy, train and evaluate the performance of a configured model. Third, we propose a method to search for configurations and demonstrate that it leads to good DNN models. We evaluate our method by applying it on image classification tasks (MNIST, CIFAR-10) and show that, with limited amount of computation and training, our method can identify high-performing architectures (with high accuracy). We also demonstrate that we outperform existing state-of-the-art architectures handcrafted by ML researchers. Our FM and framework have been released %and are publicly available to support replication and future research.