On the performance of deep learning for numerical optimization: an application to protein structure prediction
This work addresses the problem of efficiently solving global optimization problems for researchers and practitioners in computational optimization and bioinformatics, offering an incremental approach by applying NAS.
This paper explores the use of deep learning models for global optimization problems, specifically applying a neural architecture search (NAS) approach to generate lightweight neural networks. The method achieved competitive results on the CEC 2017 benchmark suite and protein structure prediction problems when compared to hand-designed algorithms, given sufficient computational resources.
Deep neural networks have recently drawn considerable attention to build and evaluate artificial learning models for perceptual tasks. Here, we present a study on the performance of the deep learning models to deal with global optimization problems. The proposed approach adopts the idea of the neural architecture search (NAS) to generate efficient neural networks for solving the problem at hand. The space of network architectures is represented using a directed acyclic graph and the goal is to find the best architecture to optimize the objective function for a new, previously unknown task. Different from proposing very large networks with GPU computational burden and long training time, we focus on searching for lightweight implementations to find the best architecture. The performance of NAS is first analyzed through empirical experiments on CEC 2017 benchmark suite. Thereafter, it is applied to a set of protein structure prediction (PSP) problems. The experiments reveal that the generated learning models can achieve competitive results when compared to hand-designed algorithms; given enough computational budget