To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networks
This addresses the need for efficient neural networks on resource-constrained devices, offering a principled method that is incremental over existing pruning techniques.
The paper tackles the problem of pruning dense neural networks to reduce size without performance loss, introducing a chaos-causality approach that identifies causal weights for misclassification, resulting in pruned networks that maintain original performance and explainability.
Reducing the size of a neural network (pruning) by removing weights without impacting its performance is an important problem for resource-constrained devices. In the past, pruning was typically accomplished by ranking or penalizing weights based on criteria like magnitude and removing low-ranked weights before retraining the remaining ones. Pruning strategies may also involve removing neurons from the network in order to achieve the desired reduction in network size. We formulate pruning as an optimization problem with the objective of minimizing misclassifications by selecting specific weights. To accomplish this, we have introduced the concept of chaos in learning (Lyapunov exponents) via weight updates and exploiting causality to identify the causal weights responsible for misclassification. Such a pruned network maintains the original performance and retains feature explainability.