Automatic Attention Pruning: Improving and Automating Model Pruning using Attentions
This addresses the need for efficient model deployment on resource-constrained edge devices by automating pruning to avoid manual tuning, though it is incremental as it builds on existing structured pruning methods.
The paper tackles the problem of compressing deep learning models for edge deployment by introducing Automatic Attention Pruning (AAP), an adaptive structured pruning method that uses attention maps to automatically prune filters, resulting in models that outperform state-of-the-art structured pruning approaches across various architectures.
Pruning is a promising approach to compress deep learning models in order to deploy them on resource-constrained edge devices. However, many existing pruning solutions are based on unstructured pruning, which yields models that cannot efficiently run on commodity hardware; and they often require users to manually explore and tune the pruning process, which is time-consuming and often leads to sub-optimal results. To address these limitations, this paper presents Automatic Attention Pruning (AAP), an adaptive, attention-based, structured pruning approach to automatically generate small, accurate, and hardware-efficient models that meet user objectives. First, it proposes iterative structured pruning using activation-based attention maps to effectively identify and prune unimportant filters. Then, it proposes adaptive pruning policies for automatically meeting the pruning objectives of accuracy-critical, memory-constrained, and latency-sensitive tasks. A comprehensive evaluation shows that AAP substantially outperforms the state-of-the-art structured pruning works for a variety of model architectures. Our code is at: https://github.com/kaiqi123/Automatic-Attention-Pruning.git.