Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis
This work provides a more interpretable method for semantic change analysis, benefiting historical linguists and lexicographers, though it is incremental in its application of existing language models.
The authors tackled the problem of creating interpretable word sense representations by generating natural language definitions for usage clusters, which they applied to semantic change analysis. They demonstrated that these definitions not only improve interpretability for users like linguists but also outperform traditional embeddings in semantic similarity tasks.
We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations. Given a collection of usage examples for a target word, and the corresponding data-driven usage clusters (i.e., word senses), a definition is generated for each usage with a specialised Flan-T5 language model, and the most prototypical definition in a usage cluster is chosen as the sense label. We demonstrate how the resulting sense labels can make existing approaches to semantic change analysis more interpretable, and how they can allow users -- historical linguists, lexicographers, or social scientists -- to explore and intuitively explain diachronic trajectories of word meaning. Semantic change analysis is only one of many possible applications of the `definitions as representations' paradigm. Beyond being human-readable, contextualised definitions also outperform token or usage sentence embeddings in word-in-context semantic similarity judgements, making them a new promising type of lexical representation for NLP.