Enhancing Certifiable Semantic Robustness via Robust Pruning of Deep Neural Networks
This work addresses the problem of scalable and tight robustness certification for deep neural networks in vision and robotics applications, representing an incremental improvement over existing methods.
The paper tackles the challenge of certifying neural network robustness against semantic perturbations like brightness and contrast by addressing over-parameterization, and it introduces a pruning method based on a new metric that achieves superior certification performance and efficiency in experiments.
Deep neural networks have been widely adopted in many vision and robotics applications with visual inputs. It is essential to verify its robustness against semantic transformation perturbations, such as brightness and contrast. However, current certified training and robustness certification methods face the challenge of over-parameterization, which hinders the tightness and scalability due to the over-complicated neural networks. To this end, we first analyze stability and variance of layers and neurons against input perturbation, showing that certifiable robustness can be indicated by a fundamental Unbiased and Smooth Neuron metric (USN). Based on USN, we introduce a novel neural network pruning method that removes neurons with low USN and retains those with high USN, thereby preserving model expressiveness without over-parameterization. To further enhance this pruning process, we propose a new Wasserstein distance loss to ensure that pruned neurons are more concentrated across layers. We validate our approach through extensive experiments on the challenging robust keypoint detection task, which involves realistic brightness and contrast perturbations, demonstrating that our method achieves superior robustness certification performance and efficiency compared to baselines.