Mitigating Architectural Mismatch During the Evolutionary Synthesis of Deep Neural Networks
This addresses an incremental improvement for researchers in evolutionary deep intelligence by exploring mating policies in network synthesis.
The study tackled the problem of architectural mismatch in evolutionary synthesis of deep neural networks by introducing a gene tagging system for alignment, finding that it resulted in slower decreases in performance accuracy and storage size but produced networks comparable in size and performance to non-aligned methods.
Evolutionary deep intelligence has recently shown great promise for producing small, powerful deep neural network models via the organic synthesis of increasingly efficient architectures over successive generations. Existing evolutionary synthesis processes, however, have allowed the mating of parent networks independent of architectural alignment, resulting in a mismatch of network structures. We present a preliminary study into the effects of architectural alignment during evolutionary synthesis using a gene tagging system. Surprisingly, the network architectures synthesized using the gene tagging approach resulted in slower decreases in performance accuracy and storage size; however, the resultant networks were comparable in size and performance accuracy to the non-gene tagging networks. Furthermore, we speculate that there is a noticeable decrease in network variability for networks synthesized with gene tagging, indicating that enforcing a like-with-like mating policy potentially restricts the exploration of the search space of possible network architectures.