Transformers perform adaptive partial pooling
This work provides insights into the learning dynamics of transformers for language modeling, which is incremental as it applies existing statistical concepts to AI models.
The paper investigates how transformers, specifically GPT2, adaptively pool evidence across contexts during training, showing that pooling decreases with training and is influenced by context frequency, number, and variability, similar to hierarchical regression.
Because language is creative, any reasonable language model must generalize, deciding what to say in novel contexts by using information from similar contexts. But what about contexts that are not novel but merely infrequent? In hierarchical regression, the model's predictions for behavior in a context are affected by observations from other similar contexts to the extent that 1) the current context is infrequent and 2) different contexts behave similarly. This is called adaptive partial pooling of evidence. This paper shows that next-word predictions of a transformer (GPT2) are increasingly unaffected by observations from outside the current context across epochs of training (the amount of pooling reduces with training), and that the extent of pooling is affected by context frequency, context number (type frequency) and context variability in a similar way to hierarchical regression. These characteristics of learning in transformers are argued to be realistic on both rational and empirical grounds.