On the Stability of Neural Networks in Deep Learning
It addresses stability issues in deep learning models, which is crucial for reliable AI applications, but the approach appears incremental as it builds on existing sensitivity analysis and regularization techniques.
This thesis tackled the problem of neural network instability and vulnerability to input perturbations and sharp loss landscapes by developing a unified framework combining Lipschitz networks, curvature regularization, and randomized smoothing, resulting in improved generalization, adversarial robustness, and training stability with contributions like efficient spectral norm computation and novel layers.
Deep learning has achieved remarkable success across a wide range of tasks, but its models often suffer from instability and vulnerability: small changes to the input may drastically affect predictions, while optimization can be hindered by sharp loss landscapes. This thesis addresses these issues through the unifying perspective of sensitivity analysis, which examines how neural networks respond to perturbations at both the input and parameter levels. We study Lipschitz networks as a principled way to constrain sensitivity to input perturbations, thereby improving generalization, adversarial robustness, and training stability. To complement this architectural approach, we introduce regularization techniques based on the curvature of the loss function, promoting smoother optimization landscapes and reducing sensitivity to parameter variations. Randomized smoothing is also explored as a probabilistic method for enhancing robustness at decision boundaries. By combining these perspectives, we develop a unified framework where Lipschitz continuity, randomized smoothing, and curvature regularization interact to address fundamental challenges in stability. The thesis contributes both theoretical analysis and practical methodologies, including efficient spectral norm computation, novel Lipschitz-constrained layers, and improved certification procedures.