Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers
This work addresses the problem of understanding human vs. model language processing for researchers in computational linguistics, but it is incremental as it builds on existing studies of context effects.
The study investigated how linguistic context affects the prediction of quantifiers like 'few' and 'all', finding that models significantly outperform humans in single-sentence contexts but only slightly better in multi-sentence contexts, with human performance improving and model performance declining with more context.
We study the role of linguistic context in predicting quantifiers (`few', `all'). We collect crowdsourced data from human participants and test various models in a local (single-sentence) and a global context (multi-sentence) condition. Models significantly out-perform humans in the former setting and are only slightly better in the latter. While human performance improves with more linguistic context (especially on proportional quantifiers), model performance suffers. Models are very effective in exploiting lexical and morpho-syntactic patterns; humans are better at genuinely understanding the meaning of the (global) context.