The Mating Rituals of Deep Neural Networks: Learning Compact Feature Representations through Sexual Evolutionary Synthesis
This work addresses architectural efficiency for deep learning practitioners, but it is incremental as it builds on existing evolutionary deep intelligence methods.
The paper tackled the problem of achieving compact feature representations in deep neural networks by introducing sexual evolutionary synthesis, which produced offspring networks from two parents, resulting in approximately double architectural efficiency on MNIST with comparable testing accuracy of ~97%.
Evolutionary deep intelligence was recently proposed as a method for achieving highly efficient deep neural network architectures over successive generations. Drawing inspiration from nature, we propose the incorporation of sexual evolutionary synthesis. Rather than the current asexual synthesis of networks, we aim to produce more compact feature representations by synthesizing more diverse and generalizable offspring networks in subsequent generations via the combination of two parent networks. Experimental results were obtained using the MNIST and CIFAR-10 datasets, and showed improved architectural efficiency and comparable testing accuracy relative to the baseline asexual evolutionary neural networks. In particular, the network synthesized via sexual evolutionary synthesis for MNIST had approximately double the architectural efficiency (cluster efficiency of 34.29X and synaptic efficiency of 258.37X) in comparison to the network synthesized via asexual evolutionary synthesis, with both networks achieving a testing accuracy of ~97%.