Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation
For practitioners pruning neural networks to very high sparsity, this work provides a diagnostic to allocate repairable damage, improving post-repair accuracy when standard allocation methods fail.
The paper introduces Relative Repairability (RR), a calibration-based diagnostic that estimates the fraction of pruning-induced damage remaining after repair, and shows that RR improves allocation over ERK and LAMP near a recoverability transition in high-sparsity pruning, with up to 2% accuracy gains on CIFAR100 ResNet18.
At very high sparsity, neural network pruning does more than decide which weights remain. It also determines where pruning induced damage is placed across the network, and whether that damage can be recovered by a fixed lightweight repair procedure. We study this problem through the lens of repair conditioned sparsity allocation. We introduce Relative Repairability (RR), a calibration based diagnostic that compares the raw activation distortion caused by layerwise pruning with the residual distortion left after channelwise variance matching repair. RR estimates the fraction of local damage that remains after repair, using only unlabeled calibration data. Across ResNet18, ResNet34, and VGG16 BN on CIFAR10 and CIFAR100, we find that RR is not a universally dominant allocation rule. Instead, it is most useful near an architecture dependent recoverability transition, where standard structural or magnitude based allocation priors begin to lose reliability but post repair recovery has not yet fully collapsed. On CIFAR100 ResNet18, a fine grained sweep shows that RR improves over ERK across the central transition band and surpasses LAMP near the upper part of this band. A projection forced ablation further shows that capped ERK can over protect projection layers, shifting excessive sparsity onto regular convolutions and reducing post repair recovery. These results suggest that high sparsity pruning should allocate not only retained weights, but also repairable damage.