Targeted Activation Penalties Help CNNs Ignore Spurious Signals
This addresses the issue of poor generalization in CNNs due to spurious signals, particularly in domains like clinical imaging where ground-truth annotations are costly, though it is an incremental improvement over existing methods.
The paper tackles the problem of neural networks relying on spurious signals in training data, which harms generalization, by proposing Targeted Activation Penalty (TAP) to control these signals in deep CNNs, reducing training times and memory usage while working with cheaper annotations from pre-trained models. It demonstrates TAP's effectiveness against state-of-the-art baselines on MNIST and clinical image datasets using four CNN architectures.
Neural networks (NNs) can learn to rely on spurious signals in the training data, leading to poor generalisation. Recent methods tackle this problem by training NNs with additional ground-truth annotations of such signals. These methods may, however, let spurious signals re-emerge in deep convolutional NNs (CNNs). We propose Targeted Activation Penalty (TAP), a new method tackling the same problem by penalising activations to control the re-emergence of spurious signals in deep CNNs, while also lowering training times and memory usage. In addition, ground-truth annotations can be expensive to obtain. We show that TAP still works well with annotations generated by pre-trained models as effective substitutes of ground-truth annotations. We demonstrate the power of TAP against two state-of-the-art baselines on the MNIST benchmark and on two clinical image datasets, using four different CNN architectures.