From feature selection to continuous optimization
This work addresses optimization bottlenecks in deep learning for researchers and practitioners, but it is incremental as it builds on existing metaheuristic and feature selection concepts.
The authors tackled the challenge of solving massive optimization problems in deep learning by proposing MaNet, a method that uses feature selection to skip irrelevant evolutionary information, achieving competitive results in accuracy and scalability on continuous optimization problems.
Metaheuristic algorithms (MAs) have seen unprecedented growth thanks to their successful applications in fields including engineering and health sciences. In this work, we investigate the use of a deep learning (DL) model as an alternative tool to do so. The proposed method, called MaNet, is motivated by the fact that most of the DL models often need to solve massive nasty optimization problems consisting of millions of parameters. Feature selection is the main adopted concepts in MaNet that helps the algorithm to skip irrelevant or partially relevant evolutionary information and uses those which contribute most to the overall performance. The introduced model is applied on several unimodal and multimodal continuous problems. The experiments indicate that MaNet is able to yield competitive results compared to one of the best hand-designed algorithms for the aforementioned problems, in terms of the solution accuracy and scalability.