Monitoring Shortcut Learning using Mutual Information
This addresses the issue of unreliable model deployment in safety-critical domains like healthcare and autonomous vehicles by providing a method to assess shortcut learning, though it is incremental as it builds on existing mutual information concepts for monitoring.
The paper tackles the problem of deep neural networks failing to generalize due to shortcut learning by proposing mutual information between learned representations and inputs as a metric to detect when networks latch onto spurious correlations during training, with experiments showing it works as a domain-agnostic monitoring tool.
The failure of deep neural networks to generalize to out-of-distribution data is a well-known problem and raises concerns about the deployment of trained networks in safety-critical domains such as healthcare, finance and autonomous vehicles. We study a particular kind of distribution shift $\unicode{x2013}$ shortcuts or spurious correlations in the training data. Shortcut learning is often only exposed when models are evaluated on real-world data that does not contain the same spurious correlations, posing a serious dilemma for AI practitioners to properly assess the effectiveness of a trained model for real-world applications. In this work, we propose to use the mutual information (MI) between the learned representation and the input as a metric to find where in training, the network latches onto shortcuts. Experiments demonstrate that MI can be used as a domain-agnostic metric for monitoring shortcut learning.