Light or Full Verb? A Minimal-Pair Dataset for Probing Phraseological Competence in Language Models
This work provides a reusable resource and evaluation framework for probing phraseological competence in language models, addressing a nuanced linguistic distinction.
The authors introduce a minimal-pair dataset to probe whether language models distinguish between light-verb and full-verb uses of frequent English verbs. Probing experiments show that models differentiate these uses even in minimal contexts and exhibit separable patterns across object types.
Frequent English verbs such as 'have' and 'make' can function either as collocates in light-verb constructions or as full lexical predicates, as in 'make a decision' vs. 'make a cake'. Whether language models represent this distinction remains unclear. We introduce a large-scale controlled dataset of minimally varying English sentence series in which the same context contains the same verb in light-verb and full-verb uses. Two probing experiments show that language models differentiate between these uses even in minimal contexts and exhibit separable patterns across object types. We release the dataset, generation code, and materials as a reusable resource. The framework supports extensions to broader contexts, additional verbs, and other languages.