PropNEAT -- Efficient GPU-Compatible Backpropagation over NeuroEvolutionary Augmenting Topology Networks
This work addresses efficiency issues in neuroevolution for researchers and practitioners, enabling faster training and potential applications in low-power contexts, though it is incremental as it adapts an existing method to new hardware.
The authors tackled the problem of slow training in NeuroEvolution of Augmenting Topologies (NEAT) by introducing PropNEAT, a GPU-compatible backpropagation method that maps genomes to a layer-based architecture, resulting in substantially faster training times and competitive performance on 58 binary classification datasets, with per-epoch time scaling linearly with network depth.
We introduce PropNEAT, a fast backpropagation implementation of NEAT that uses a bidirectional mapping of the genome graph to a layer-based architecture that preserves the NEAT genomes whilst enabling efficient GPU backpropagation. We test PropNEAT on 58 binary classification datasets from the Penn Machine Learning Benchmarks database, comparing the performance against logistic regression, dense neural networks and random forests, as well as a densely retrained variant of the final PropNEAT model. PropNEAT had the second best overall performance, behind Random Forest, though the difference between the models was not statistically significant apart from between Random Forest in comparison with logistic regression and the PropNEAT retrain models. PropNEAT was substantially faster than a naive backpropagation method, and both were substantially faster and had better performance than the original NEAT implementation. We demonstrate that the per-epoch training time for PropNEAT scales linearly with network depth, and is efficient on GPU implementations for backpropagation. This implementation could be extended to support reinforcement learning or convolutional networks, and is able to find sparser and smaller networks with potential for applications in low-power contexts.