Black Big Boxes: Tracing Adjective Order Preferences in Large Language Models
This work addresses how language models acquire linguistic patterns, providing insights into their learning mechanisms, but it is incremental as it builds on existing linguistic and computational studies.
The study investigated whether large language models' adjective order preferences stem from distributional learning or go beyond surface patterns, finding that while training data frequencies explain much of their behavior, models also generalize to unseen combinations and use contextual cues.
In English and other languages, multiple adjectives in noun phrases follow intricate ordering patterns. These patterns have been widely studied in linguistics and provide a useful test case for assessing how language models (LMs) acquire graded and context-sensitive word order preferences. We ask to what extent adjective order preferences in LMs can be explained by distributional learning alone, and where models exhibit behaviour that goes beyond surface co-occurrence patterns. We find that LM predictions are largely explained by training data frequencies: simple n-gram statistics account for much of their behaviour and closely mirror the preferences learned during training. However, by analysing learning dynamics we reveal that models also generalize robustly to unseen adjective combinations, indicating that their behaviour cannot be reduced to memorization of observed orders alone. Moreover, we show how LMs leverage word order cues from sentence context, demonstrating with feature attribution methods that contextual cues are an additional driver of adjective order in LM output.