Budget-Aware Graph Convolutional Network Design using Probabilistic Magnitude Pruning
This work addresses the need for lightweight GCNs for edge devices in image processing, offering an incremental improvement over existing pruning methods.
The paper tackles the problem of oversized graph convolutional networks (GCNs) for edge deployment by proposing Probabilistic Magnitude Pruning (PMP), a method that jointly trains network topology and weights, resulting in substantial gains in skeleton-based recognition tasks, especially at high pruning rates.
Graph convolutional networks (GCNs) are nowadays becoming mainstream in solving many image processing tasks including skeleton-based recognition. Their general recipe consists in learning convolutional and attention layers that maximize classification performances. With multi-head attention, GCNs are highly accurate but oversized, and their deployment on edge devices requires their pruning. Among existing methods, magnitude pruning (MP) is relatively effective but its design is clearly suboptimal as network topology selection and weight retraining are achieved independently. In this paper, we devise a novel lightweight GCN design dubbed as Probabilistic Magnitude Pruning (PMP) that jointly trains network topology and weights. Our method is variational and proceeds by aligning the weight distribution of the learned networks with an a priori distribution. This allows implementing any fixed pruning rate, and also enhancing the generalization performances of the designed lightweight GCNs. Extensive experiments conducted on the challenging task of skeleton-based recognition show a substantial gain of our lightweight GCNs particularly at very high pruning regimes.