REPrune: Channel Pruning via Kernel Representative Selection
This work addresses the challenge of reducing accuracy loss during model compression for CNN deployment, offering an incremental improvement over existing channel pruning methods.
The paper tackles the problem of undesirable accuracy drops in channel pruning for CNNs by proposing REPrune, a technique that emulates kernel pruning to exploit finer granularity, achieving better performance in computer vision tasks with a balance between acceleration and accuracy retention.
Channel pruning is widely accepted to accelerate modern convolutional neural networks (CNNs). The resulting pruned model benefits from its immediate deployment on general-purpose software and hardware resources. However, its large pruning granularity, specifically at the unit of a convolution filter, often leads to undesirable accuracy drops due to the inflexibility of deciding how and where to introduce sparsity to the CNNs. In this paper, we propose REPrune, a novel channel pruning technique that emulates kernel pruning, fully exploiting the finer but structured granularity. REPrune identifies similar kernels within each channel using agglomerative clustering. Then, it selects filters that maximize the incorporation of kernel representatives while optimizing the maximum cluster coverage problem. By integrating with a simultaneous training-pruning paradigm, REPrune promotes efficient, progressive pruning throughout training CNNs, avoiding the conventional train-prune-finetune sequence. Experimental results highlight that REPrune performs better in computer vision tasks than existing methods, effectively achieving a balance between acceleration ratio and performance retention.