Vec2Gloss: definition modeling leveraging contextualized vectors with Wordnet gloss
This work addresses interpretability challenges in NLP for researchers and practitioners, but it is incremental as it builds on existing definition modeling and contextualized embeddings.
The authors tackled the problem of interpreting contextualized embeddings by proposing Vec2Gloss, a model that generates human-readable definitions from these embeddings, using Chinese Wordnet for gloss patterns. The results show that this approach provides a new perspective for lexical-semantic applications.
Contextualized embeddings are proven to be powerful tools in multiple NLP tasks. Nonetheless, challenges regarding their interpretability and capability to represent lexical semantics still remain. In this paper, we propose that the task of definition modeling, which aims to generate the human-readable definition of the word, provides a route to evaluate or understand the high dimensional semantic vectors. We propose a `Vec2Gloss' model, which produces the gloss from the target word's contextualized embeddings. The generated glosses of this study are made possible by the systematic gloss patterns provided by Chinese Wordnet. We devise two dependency indices to measure the semantic and contextual dependency, which are used to analyze the generated texts in gloss and token levels. Our results indicate that the proposed `Vec2Gloss' model opens a new perspective to the lexical-semantic applications of contextualized embeddings.