Feature Collapse
This addresses the problem of understanding representation learning mechanisms in neural networks, particularly for NLP researchers, and is incremental as it builds on existing concepts of feature similarity.
The paper formalizes and studies 'feature collapse', where entities with similar roles in a learning task receive similar representations, using an NLP task to show it correlates with generalization and proving that identical roles lead to identical features in neural networks in the large sample limit.
We formalize and study a phenomenon called feature collapse that makes precise the intuitive idea that entities playing a similar role in a learning task receive similar representations. As feature collapse requires a notion of task, we leverage a simple but prototypical NLP task to study it. We start by showing experimentally that feature collapse goes hand in hand with generalization. We then prove that, in the large sample limit, distinct words that play identical roles in this NLP task receive identical local feature representations in a neural network. This analysis reveals the crucial role that normalization mechanisms, such as LayerNorm, play in feature collapse and in generalization.