Augmenting Novelty Search with a Surrogate Model to Engineer Meta-Diversity in Ensembles of Classifiers
This work addresses a computational bottleneck for researchers and practitioners using neuroevolution and novelty search to create high-performing ensembles, making diversity searches more tractable with limited resources.
The paper tackles the computational cost of training evolved neural network architectures during novelty search for building diverse classifier ensembles by introducing a surrogate model to estimate behavioral distances, achieving a 10x speedup and improved results on CIFAR-10, CIFAR-100, and SVHN datasets.
Using Neuroevolution combined with Novelty Search to promote behavioural diversity is capable of constructing high-performing ensembles for classification. However, using gradient descent to train evolved architectures during the search can be computationally prohibitive. Here we propose a method to overcome this limitation by using a surrogate model which estimates the behavioural distance between two neural network architectures required to calculate the sparseness term in Novelty Search. We demonstrate a speedup of 10 times over previous work and significantly improve on previous reported results on three benchmark datasets from Computer Vision -- CIFAR-10, CIFAR-100, and SVHN. This results from the expanded architecture search space facilitated by using a surrogate. Our method represents an improved paradigm for implementing horizontal scaling of learning algorithms by making an explicit search for diversity considerably more tractable for the same bounded resources.