Sparsity Emerges Naturally in Neural Language Models
This addresses interpretability and efficiency concerns in NLP by revealing inherent sparsity patterns, though it is incremental as it builds on existing sparsity research.
The study investigated whether neural language models naturally develop sparsity during training, finding that frequent input words lead to sparse activations, while frequent target words result in dispersed activations but concentrated gradients, with function words showing more concentrated gradients than content words.
Concerns about interpretability, computational resources, and principled inductive priors have motivated efforts to engineer sparse neural models for NLP tasks. If sparsity is important for NLP, might well-trained neural models naturally become roughly sparse? Using the Taxi-Euclidean norm to measure sparsity, we find that frequent input words are associated with concentrated or sparse activations, while frequent target words are associated with dispersed activations but concentrated gradients. We find that gradients associated with function words are more concentrated than the gradients of content words, even controlling for word frequency.