Adversarial robustness through Lipschitz-Guided Stochastic Depth in Neural Networks
This work addresses adversarial robustness for computer vision models, offering a defense with reduced computational cost, but it is incremental as it builds on existing stochastic depth techniques.
The paper tackles the vulnerability of deep neural networks and Vision Transformers to adversarial perturbations by proposing a Lipschitz-guided stochastic depth method that controls the effective Lipschitz constant. The result shows maintained near-baseline clean accuracy, enhanced robustness under attacks like FGSM, PGD-20, and AutoAttack, and significant FLOPs reduction on CIFAR-10 with ViT-Tiny.
Deep neural networks and Vision Transformers achieve state-of-the-art performance in computer vision but are highly vulnerable to adversarial perturbations. Standard defenses often incur high computational cost or lack formal guarantees. We propose a Lipschitz-guided stochastic depth (DropPath) method, where drop probabilities increase with depth to control the effective Lipschitz constant of the network. This approach regularizes deeper layers, improving robustness while preserving clean accuracy and reducing computation. Experiments on CIFAR-10 with ViT-Tiny show that our custom depth-dependent schedule maintains near-baseline clean accuracy, enhances robustness under FGSM, PGD-20, and AutoAttack, and significantly reduces FLOPs compared to baseline and linear DropPath schedules.