Neuroevolution in Deep Learning: The Role of Neutrality
This work addresses the problem of high computational costs in automated neural network optimization for researchers and practitioners, but it appears incremental as it builds on existing neuroevolution concepts.
The paper tackles the computational expense of neuroevolution for deep neural networks by exploring how neutrality, inspired by Kimura's neutral theory, can accelerate training and design under certain conditions, though no concrete numerical results are provided.
A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation of DNNs. Neuroevolution is a term which describes these processes of automated configuration and training of DNNs using EAs. However, the automatic design and/or training of these modern neural networks through evolutionary algorithms is computanalli expensive. Kimura's neutral theory of molecular evolution states that the majority of evolutionary changes at molecular level are the result of random fixation of selectively neutral mutations. A mutation from one gene to another is neutral if it does not affect the phenotype. This work discusses how neutrality, given certain conditions, can help to speed up the training/design of deep neural networks.