Computational Sentence-level Metrics Predicting Human Sentence Comprehension
This work addresses the gap in computational psycholinguistics for sentence-level analysis, offering interpretable metrics that could integrate LLMs with cognitive science, though it is incremental in applying existing models to a new task.
The study tackled the problem of predicting human sentence comprehension by developing sentence-level metrics like surprisal and relevance using multilingual large language models, achieving high accuracy in predicting reading speeds across languages.
The majority of research in computational psycholinguistics has concentrated on the processing of words. This study introduces innovative methods for computing sentence-level metrics using multilingual large language models. The metrics developed sentence surprisal and sentence relevance and then are tested and compared to validate whether they can predict how humans comprehend sentences as a whole across languages. These metrics offer significant interpretability and achieve high accuracy in predicting human sentence reading speeds. Our results indicate that these computational sentence-level metrics are exceptionally effective at predicting and elucidating the processing difficulties encountered by readers in comprehending sentences as a whole across a variety of languages. Their impressive performance and generalization capabilities provide a promising avenue for future research in integrating LLMs and cognitive science.