Unseen Word Representation by Aligning Heterogeneous Lexical Semantic Spaces
This addresses the issue of data sparsity for rare words in NLP, benefiting tasks like text classification, though it is incremental as it adapts existing graph embedding and cross-lingual techniques.
The paper tackles the problem of representing unseen or rare words in word embeddings by leveraging lexical resources like WordNet, resulting in consistent performance improvements across rare word similarity datasets and downstream text classification tasks.
Word embedding techniques heavily rely on the abundance of training data for individual words. Given the Zipfian distribution of words in natural language texts, a large number of words do not usually appear frequently or at all in the training data. In this paper we put forward a technique that exploits the knowledge encoded in lexical resources, such as WordNet, to induce embeddings for unseen words. Our approach adapts graph embedding and cross-lingual vector space transformation techniques in order to merge lexical knowledge encoded in ontologies with that derived from corpus statistics. We show that the approach can provide consistent performance improvements across multiple evaluation benchmarks: in-vitro, on multiple rare word similarity datasets, and in-vivo, in two downstream text classification tasks.