Finding Syntax in Human Encephalography with Beam Search
This provides a mechanistic model for syntactic processing in human language comprehension, which is incremental as it builds on existing methods to link computational models to neural data.
The paper tackled the problem of modeling syntactic processing in human language comprehension by using recurrent neural network grammars (RNNGs) with beam search to analyze electrophysiological responses, finding two amplitude effects (an early peak and a P600-like later peak) that were attributed to syntactic composition, while a non-syntactic model showed no reliable effects.
Recurrent neural network grammars (RNNGs) are generative models of (tree,string) pairs that rely on neural networks to evaluate derivational choices. Parsing with them using beam search yields a variety of incremental complexity metrics such as word surprisal and parser action count. When used as regressors against human electrophysiological responses to naturalistic text, they derive two amplitude effects: an early peak and a P600-like later peak. By contrast, a non-syntactic neural language model yields no reliable effects. Model comparisons attribute the early peak to syntactic composition within the RNNG. This pattern of results recommends the RNNG+beam search combination as a mechanistic model of the syntactic processing that occurs during normal human language comprehension.