Pruning a neural network using Bayesian inference
This work addresses the problem of reducing computational and memory demands in neural networks for AI practitioners, but it is incremental as it builds on existing pruning techniques with a Bayesian twist.
The paper tackles neural network pruning by introducing a Bayesian inference approach that uses Bayes factors to guide iterative pruning, achieving competitive accuracy with desired sparsity levels on multiple benchmarks.
Neural network pruning is a highly effective technique aimed at reducing the computational and memory demands of large neural networks. In this research paper, we present a novel approach to pruning neural networks utilizing Bayesian inference, which can seamlessly integrate into the training procedure. Our proposed method leverages the posterior probabilities of the neural network prior to and following pruning, enabling the calculation of Bayes factors. The calculated Bayes factors guide the iterative pruning. Through comprehensive evaluations conducted on multiple benchmarks, we demonstrate that our method achieves desired levels of sparsity while maintaining competitive accuracy.