Growing Artificial Neural Networks
This addresses the need for efficient neural network deployment in embedded hardware, offering a novel approach to size reduction.
The paper tackles the problem of reducing neural network size for low SWaP hardware by proposing an algorithm that grows networks instead of pruning them, achieving 98.80% test accuracy with only 21,211 weights compared to 98.74% with 61,160 weights.
Pruning is a legitimate method for reducing the size of a neural network to fit in low SWaP hardware, but the networks must be trained and pruned offline. We propose an algorithm, Artificial Neurogenesis (ANG), that grows rather than prunes the network and enables neural networks to be trained and executed in low SWaP embedded hardware. ANG accomplishes this by using the training data to determine critical connections between layers before the actual training takes place. Our experiments use a modified LeNet-5 as a baseline neural network that achieves a test accuracy of 98.74% using a total of 61,160 weights. An ANG grown network achieves a test accuracy of 98.80% with only 21,211 weights.