Learning explanations that are hard to vary
This addresses the issue of unreliable explanations in deep learning for researchers and practitioners, though it appears incremental as it builds on existing regularization concepts.
The paper tackles the problem of deep learning models memorizing or using patchwork solutions instead of identifying invariant patterns, by proposing an algorithm based on a logical AND to focus on invariances and prevent memorization in real-world tasks.
In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and `patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND, that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.