Quantum Neuron Selection: Finding High Performing Subnetworks With Quantum Algorithms
This addresses the challenge of efficient neural network training and pruning for AI researchers, but it is incremental as it builds on existing insights about subnetworks and focuses on quantum method formulation.
The paper tackles the problem of finding high-performing subnetworks within large, randomly initialized neural networks, which is combinatorially hard for classical algorithms, by exploring quantum algorithms to make this more tractable on current hardware.
Gradient descent methods have long been the de facto standard for training deep neural networks. Millions of training samples are fed into models with billions of parameters, which are slowly updated over hundreds of epochs. Recently, it's been shown that large, randomly initialized neural networks contain subnetworks that perform as well as fully trained models. This insight offers a promising avenue for training future neural networks by simply pruning weights from large, random models. However, this problem is combinatorically hard and classical algorithms are not efficient at finding the best subnetwork. In this paper, we explore how quantum algorithms could be formulated and applied to this neuron selection problem. We introduce several methods for local quantum neuron selection that reduce the entanglement complexity that large scale neuron selection would require, making this problem more tractable for current quantum hardware.