Integrating large language models and active inference to understand eye movements in reading and dyslexia
This work addresses understanding cognitive processes in reading and dyslexia, offering a novel computational approach with potential implications for studying maladaptive inference in dyslexic individuals, though it appears incremental as it combines existing methods.
The researchers tackled the problem of simulating reading and eye movements by developing a computational model that integrates large language models with hierarchical active inference, showing that attenuating priors leads to more fragmented reading with shorter saccades, aligning with dyslexic eye movement patterns.
We present a novel computational model employing hierarchical active inference to simulate reading and eye movements. The model characterizes linguistic processing as inference over a hierarchical generative model, facilitating predictions and inferences at various levels of granularity, from syllables to sentences. Our approach combines the strengths of large language models for realistic textual predictions and active inference for guiding eye movements to informative textual information, enabling the testing of predictions. The model exhibits proficiency in reading both known and unknown words and sentences, adhering to the distinction between lexical and nonlexical routes in dual route theories of reading. Our model therefore provides a novel approach to understand the cognitive processes underlying reading and eye movements, within a predictive processing framework. Furthermore, our model can potentially aid in understanding how maladaptive predictive processing can produce reading deficits associated with dyslexia. As a proof of concept, we show that attenuating the contribution of priors during the reading process leads to incorrect inferences and a more fragmented reading style, characterized by a greater number of shorter saccades, aligning with empirical findings regarding eye movements in dyslexic individuals. In summary, our model represents a significant advancement in comprehending the cognitive processes involved in reading and eye movements, with potential implications for understanding dyslexia in terms of maladaptive inference.