Regularized Evolutionary Algorithm for Dynamic Neural Topology Search
This work addresses memory efficiency in neuro-evolutionary architecture search, which is an incremental improvement for researchers in automated machine learning.
The paper tackles the memory expense of evolutionary algorithms for neural architecture search by proposing a Regularized Evolutionary Algorithm with a micro-population of 10 individuals, achieving competitive results on digits datasets like MNIST, USPS, and SVHN.
Designing neural networks for object recognition requires considerable architecture engineering. As a remedy, neuro-evolutionary network architecture search, which automatically searches for optimal network architectures using evolutionary algorithms, has recently become very popular. Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i.e., network architectures) and is therefore memory expensive. In this work, we propose a Regularized Evolutionary Algorithm with low memory footprint to evolve a dynamic image classifier. In details, we introduce novel custom operators that regularize the evolutionary process of a micro-population of 10 individuals. We conduct experiments on three different digits datasets (MNIST, USPS, SVHN) and show that our evolutionary method obtains competitive results with the current state-of-the-art.