A unified information-theoretic model of EEG signatures of human language processing
This work addresses the challenge of understanding neural correlates of language processing for cognitive neuroscience and computational linguistics, offering a precise formulation that integrates with existing theories.
The authors tackled the problem of modeling human language processing in the brain by proposing an information-theoretic model that decomposes word surprisal into heuristic surprise and discrepancy signal, corresponding to EEG signatures N400 and P600, and validated it by simulating ERP patterns from experimental data.
We advance an information-theoretic model of human language processing in the brain, in which incoming linguistic input is processed at two levels, in terms of a heuristic interpretation and in terms of error correction. We propose that these two kinds of information processing have distinct electroencephalographic signatures, corresponding to the well-documented N400 and P600 components of language-related event-related potentials (ERPs). Formally, we show that the information content (surprisal) of a word in context can be decomposed into two quantities: (A) heuristic surprise, which signals processing difficulty of word given its inferred context, and corresponds with the N400 signal; and (B) discrepancy signal, which reflects divergence between the true context and the inferred context, and corresponds to the P600 signal. Both of these quantities can be estimated using modern NLP techniques. We validate our theory by successfully simulating ERP patterns elicited by a variety of linguistic manipulations in previously-reported experimental data from Ryskin et al. (2021). Our theory is in principle compatible with traditional cognitive theories assuming a `good-enough' heuristic interpretation stage, but with precise information-theoretic formulation.