One-Shot Pruning for Fast-adapting Pre-trained Models on Devices
This work addresses the problem of efficient model deployment on resource-constrained devices, offering a scalable solution for downstream tasks, though it is incremental as it builds on existing pruning techniques.
The paper tackles the challenge of deploying large pre-trained models on low-capability devices by proposing a one-shot pruning method that uses knowledge from similar tasks to extract sub-networks, achieving better accuracy and efficiency than baseline methods across CNNs and ViTs on various datasets.
Large-scale pre-trained models have been remarkably successful in resolving downstream tasks. Nonetheless, deploying these models on low-capability devices still requires an effective approach, such as model pruning. However, pruning the model from scratch can pose a practical challenge given the limited resources of each downstream task or device. To tackle this issue, we present a scalable one-shot pruning method that leverages pruned knowledge of similar tasks to extract a sub-network from the pre-trained model for a new task. Specifically, we create a score mask using the pruned models of similar tasks to identify task-specific filters/nodes in the pre-trained model for the new task. Based on this mask, we conduct a single round of pruning to extract a suitably-sized sub-network that can quickly adapt to the new task with only a few training iterations. Our experimental analysis demonstrates the effectiveness of the proposed method on the convolutional neural networks (CNNs) and vision transformers (ViT) with various datasets. The proposed method consistently outperforms popular pruning baseline methods in terms of accuracy and efficiency when dealing with diverse downstream tasks with different memory constraints.