Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?
This work addresses a cognitive science problem by explaining the superior performance of transformers in modeling human language comprehension metrics, though it is incremental in nature.
The paper investigates why transformer language models outperform recurrent neural networks in predicting N400 amplitude, a neural signal linked to human language processing difficulty, and provides evidence that transformers' predictions are influenced by preceding context similarly to semantic facilitation in humans.
Despite being designed for performance rather than cognitive plausibility, transformer language models have been found to be better at predicting metrics used to assess human language comprehension than language models with other architectures, such as recurrent neural networks. Based on how well they predict the N400, a neural signal associated with processing difficulty, we propose and provide evidence for one possible explanation - their predictions are affected by the preceding context in a way analogous to the effect of semantic facilitation in humans.