Definition Modeling: Learning to define word embeddings in natural language
This addresses the need for more transparent semantic evaluation in NLP, though it is incremental in proposing a new task and model variants.
The paper tackles the problem of generating dictionary definitions from word embeddings as a more direct evaluation of lexical semantics, and finds that a model controlling dependencies between the word and definition words performs significantly better, with a character-level convolution layer improving results.
Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings' semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a character-level convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.