Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives
This work addresses the problem of improving models of human language comprehension for researchers in psycholinguistics and NLP, though it is incremental as it builds on existing surprisal-based methods.
The paper tackles the problem of measuring utterance predictability by introducing 'information value', a metric that quantifies predictability relative to plausible alternatives, and finds it is a stronger predictor of utterance acceptability than token-level surprisal and complementary for predicting reading times.
We present information value, a measure which quantifies the predictability of an utterance relative to a set of plausible alternatives. We introduce a method to obtain interpretable estimates of information value using neural text generators, and exploit their psychometric predictive power to investigate the dimensions of predictability that drive human comprehension behaviour. Information value is a stronger predictor of utterance acceptability in written and spoken dialogue than aggregates of token-level surprisal and it is complementary to surprisal for predicting eye-tracked reading times.