Partition Pruning: Parallelization-Aware Pruning for Deep Neural Networks
This addresses memory and speed issues for deploying neural networks on accelerators, but it appears incremental as it builds on existing pruning techniques with a focus on parallelization.
The paper tackles the problem of high memory usage and slow inference in deep neural networks by developing Partition Pruning, a scheme that reduces parameters while considering parallelization, resulting in a 7.72x speedup and 2.73x energy reduction for pruned layers of TinyVGG16 compared to the unpruned model on a single accelerator.
Parameters of recent neural networks require a huge amount of memory. These parameters are used by neural networks to perform machine learning tasks when processing inputs. To speed up inference, we develop Partition Pruning, an innovative scheme to reduce the parameters used while taking into consideration parallelization. We evaluated the performance and energy consumption of parallel inference of partitioned models, which showed a 7.72x speed up of performance and a 2.73x reduction in the energy used for computing pruned layers of TinyVGG16 in comparison to running the unpruned model on a single accelerator. In addition, our method showed a limited reduction some numbers in accuracy while partitioning fully connected layers.