Efficient Multi-Prize Lottery Tickets: Enhanced Accuracy, Training, and Inference Speed
This work addresses limitations in compact neural network design for efficient AI deployment, but it is incremental as it builds on existing methods.
The paper tackled the sensitivity to prune ratio and lack of training/inference speed benefits in multi-prize lottery tickets for binary neural networks, achieving improved accuracy and speed on CIFAR-10.
Recently, Diffenderfer and Kailkhura proposed a new paradigm for learning compact yet highly accurate binary neural networks simply by pruning and quantizing randomly weighted full precision neural networks. However, the accuracy of these multi-prize tickets (MPTs) is highly sensitive to the optimal prune ratio, which limits their applicability. Furthermore, the original implementation did not attain any training or inference speed benefits. In this report, we discuss several improvements to overcome these limitations. We show the benefit of the proposed techniques by performing experiments on CIFAR-10.