Systematic word meta-sense extension
This addresses a challenge in automated processing of non-literal language for NLP applications, but it is incremental as it builds on existing analogy and language model techniques.
The paper tackles the problem of language models' limited ability to extend word meanings to new semantic domains, such as metaphors, by introducing the SWORME task and showing that an analogy-based method improves systematicity and benefits figurative language understanding benchmarks.
The meaning of polysemous words often varies in a highly productive yet predictable way. Generalizing the regularity between conventional senses to derive novel word meaning is crucial for automated processing of non-literal language uses such as figurative expressions. We introduce a novel task called systematic word meta-sense extension (SWORME) to test and improve language models' ability to extend word meaning to denote new semantic domains (also called meta-senses) that bear regular semantic relations with existing senses. We found that language models prefer incremental lexical semantic change toward conceptually similar meta-senses such as logical metonymy, and are much worse at predicting highly non-literal meaning extensions such as metaphors. We propose a novel analogy-based method of word meaning extension, and show that it effectively improves language model systematicity in making both gradual and radical types of meta-sense extension. We further demonstrate that learning systematic meta-sense extensions benefits language models on multiple benchmarks of figurative language understanding.