Simplicity Bias in 1-Hidden Layer Neural Networks
This work addresses the issue of neural networks' tendency to rely on simple, potentially non-robust features, which is a problem for improving model robustness and generalization in machine learning, though it is incremental as it builds on prior semi-synthetic studies.
The paper tackles the problem of neural networks exhibiting simplicity bias (SB), where they learn only the simplest features, by rigorously defining and establishing SB for one-hidden-layer networks, showing theoretically that networks depend on low-dimensional projections even with more complex features available, and empirically demonstrating this on real datasets like Imagenette and Waterbirds-Landbirds.
Recent works have demonstrated that neural networks exhibit extreme simplicity bias(SB). That is, they learn only the simplest features to solve a task at hand, even in the presence of other, more robust but more complex features. Due to the lack of a general and rigorous definition of features, these works showcase SB on semi-synthetic datasets such as Color-MNIST, MNIST-CIFAR where defining features is relatively easier. In this work, we rigorously define as well as thoroughly establish SB for one hidden layer neural networks. More concretely, (i) we define SB as the network essentially being a function of a low dimensional projection of the inputs (ii) theoretically, we show that when the data is linearly separable, the network primarily depends on only the linearly separable ($1$-dimensional) subspace even in the presence of an arbitrarily large number of other, more complex features which could have led to a significantly more robust classifier, (iii) empirically, we show that models trained on real datasets such as Imagenette and Waterbirds-Landbirds indeed depend on a low dimensional projection of the inputs, thereby demonstrating SB on these datasets, iv) finally, we present a natural ensemble approach that encourages diversity in models by training successive models on features not used by earlier models, and demonstrate that it yields models that are significantly more robust to Gaussian noise.