Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs
This work addresses the need for reduced engineering effort in neural network design for AI researchers, though it is an incremental step toward mimicking biological development.
The paper tackles the problem of designing neural architectures by proposing a self-organizing growth process guided by a Neural Developmental Program (NDP), which operates through local communication, and investigates its role across various machine learning benchmarks and optimization methods.
Biological nervous systems are created in a fundamentally different way than current artificial neural networks. Despite its impressive results in a variety of different domains, deep learning often requires considerable engineering effort to design high-performing neural architectures. By contrast, biological nervous systems are grown through a dynamic self-organizing process. In this paper, we take initial steps toward neural networks that grow through a developmental process that mirrors key properties of embryonic development in biological organisms. The growth process is guided by another neural network, which we call a Neural Developmental Program (NDP) and which operates through local communication alone. We investigate the role of neural growth on different machine learning benchmarks and different optimization methods (evolutionary training, online RL, offline RL, and supervised learning). Additionally, we highlight future research directions and opportunities enabled by having self-organization driving the growth of neural networks.