Structure-Aware Automatic Channel Pruning by Searching with Graph Embedding
This work addresses efficient deployment of neural networks on resource-constrained devices by improving pruning decisions, though it is incremental as it builds on existing pruning methods with a novel structural approach.
The paper tackles the problem of suboptimal channel pruning in deep neural networks by proposing a structure-aware automatic pruning framework that uses graph convolutional networks to model global dependencies, achieving improved compression efficiency and competitive accuracy on benchmark datasets like CIFAR-10 and ImageNet.
Channel pruning is a powerful technique to reduce the computational overhead of deep neural networks, enabling efficient deployment on resource-constrained devices. However, existing pruning methods often rely on local heuristics or weight-based criteria that fail to capture global structural dependencies within the network, leading to suboptimal pruning decisions and degraded model performance. To address these limitations, we propose a novel structure-aware automatic channel pruning (SACP) framework that utilizes graph convolutional networks (GCNs) to model the network topology and learn the global importance of each channel. By encoding structural relationships within the network, our approach implements topology-aware pruning and this pruning is fully automated, reducing the need for human intervention. We restrict the pruning rate combinations to a specific space, where the number of combinations can be dynamically adjusted, and use a search-based approach to determine the optimal pruning rate combinations. Extensive experiments on benchmark datasets (CIFAR-10, ImageNet) with various models (ResNet, VGG16) demonstrate that SACP outperforms state-of-the-art pruning methods on compression efficiency and competitive on accuracy retention.