Evaluating the cognitive reality of Spanish irregular morphomic patterns: Humans vs. Transformers
This work addresses the cognitive plausibility of AI models in linguistics, showing incremental insights into how transformers differ from human language processing.
The study compared transformer models to human data on Spanish irregular morphomic patterns, finding that while models achieved higher accuracy in stem and suffix prediction, they diverged from humans by preferring irregular responses and being influenced by training data frequency, with sensitivity to phonological similarity only in certain conditions.
This study investigates the cognitive plausibility of the Spanish irregular morphomic pattern by directly comparing transformer-based neural networks to human behavioral data from \citet{Nevins2015TheRA}. Using the same analytical framework as the original human study, we evaluate whether transformer models can replicate human-like sensitivity to a complex linguistic phenomena, the morphome, under controlled input conditions. Our experiments focus on three frequency conditions: natural, low-frequency, and high-frequency distributions of verbs exhibiting irregular morphomic patterns. While the models outperformed humans in stem and suffix accuracy, a clear divergence emerged in response preferences. Unlike humans, who consistently favored natural responses across all test items, models' preferred irregular responses and were influenced by the proportion of irregular verbs in their training data. Additionally, models trained on the natural and low-frequency distributions, but not the high-frequency distribution, were sensitive to the phonological similarity between test items and real Spanish L-shaped verbs.