A New Framework to Analyse the Distributional Robustness of Deep Neural Networks
Provides model-level diagnostics for robustness to distribution shifts, but the approach is incremental and lacks quantitative comparisons to existing methods.
The paper proposes a framework to analyze distributional robustness of DNNs by modeling layer weight-activation interactions with Bernoulli distributions, using class separation as a proxy. Metrics distinguish memorized vs. non-memorized networks on CIFAR-10 and ImageNet, and show reduced separation under distribution shifts.
Deep neural networks have achieved impressive performance on a variety of tasks, but their brittleness to distributional shifts remains a significant barrier to real-world deployment. In this paper, we propose a framework to analyse and quantify the distributional robustness of neural networks by studying the interactions between layer weights and activations. We model these interactions using Bernoulli distributions, using the separation between classes as a diagnostic proxy for robustness. We demonstrate the usefulness of this framework through models trained on CIFAR-10 and ImageNet. We show that our proposed metrics can distinguish between networks that have memorised their training data and those that have not. We also perform analogous experiments in the activation space and find that the same properties do not hold up. Additionally, we investigate the behaviour of our metrics under various distribution shifts and show that these shifts reduce separation under our path-based diagnostics. Our results suggest that this framework provides useful model-level diagnostics of representation structure and robustness.