Architecture Agnostic Neural Networks
This work addresses the challenge of reducing reliance on hand-crafted architectures in AI, though it appears incremental as it builds on existing sparse and binary network paradigms.
The paper tackled the problem of building artificial neural networks that are architecture agnostic, inspired by biological neural networks, by creating families of sparse, binary networks not trained via backpropagation, achieving high performance in static and dynamic tasks.
In this paper, we explore an alternate method for synthesizing neural network architectures, inspired by the brain's stochastic synaptic pruning. During a person's lifetime, numerous distinct neuronal architectures are responsible for performing the same tasks. This indicates that biological neural networks are, to some degree, architecture agnostic. However, artificial networks rely on their fine-tuned weights and hand-crafted architectures for their remarkable performance. This contrast begs the question: Can we build artificial architecture agnostic neural networks? To ground this study we utilize sparse, binary neural networks that parallel the brain's circuits. Within this sparse, binary paradigm we sample many binary architectures to create families of architecture agnostic neural networks not trained via backpropagation. These high-performing network families share the same sparsity, distribution of binary weights, and succeed in both static and dynamic tasks. In summation, we create an architecture manifold search procedure to discover families or architecture agnostic neural networks.