Incremental Evolution and Development of Deep Artificial Neural Networks
This work addresses the inefficiency of evolutionary processes in neural network design for non-expert users, though it is incremental as it builds on an existing method.
The authors tackled the problem of NeuroEvolution methods starting from scratch for each task by extending Fast-DENSER to incremental development, which transfers knowledge from previous tasks, resulting in statistically superior average performance, similar performance with fewer evaluations, and better generalization to unseen problems.
NeuroEvolution (NE) methods are known for applying Evolutionary Computation to the optimisation of Artificial Neural Networks(ANNs). Despite aiding non-expert users to design and train ANNs, the vast majority of NE approaches disregard the knowledge that is gathered when solving other tasks, i.e., evolution starts from scratch for each problem, ultimately delaying the evolutionary process. To overcome this drawback, we extend Fast Deep Evolutionary Network Structured Representation (Fast-DENSER) to incremental development. We hypothesise that by transferring the knowledge gained from previous tasks we can attain superior results and speedup evolution. The results show that the average performance of the models generated by incremental development is statistically superior to the non-incremental average performance. In case the number of evaluations performed by incremental development is smaller than the performed by non-incremental development the attained results are similar in performance, which indicates that incremental development speeds up evolution. Lastly, the models generated using incremental development generalise better, and thus, without further evolution, report a superior performance on unseen problems.