Misspelling Oblivious Word Embeddings
This addresses the issue of limited applicability of word embeddings to malformed texts for NLP practitioners, though it appears incremental as it builds on FastText with subwords and supervised learning.
The paper tackles the problem of word embeddings being ineffective with misspelled text by proposing a method that learns embeddings resilient to misspellings, embedding misspellings close to their correct variants, and demonstrates advantages on NLP tasks using public test sets.
In this paper we present a method to learn word embeddings that are resilient to misspellings. Existing word embeddings have limited applicability to malformed texts, which contain a non-negligible amount of out-of-vocabulary words. We propose a method combining FastText with subwords and a supervised task of learning misspelling patterns. In our method, misspellings of each word are embedded close to their correct variants. We train these embeddings on a new dataset we are releasing publicly. Finally, we experimentally show the advantages of this approach on both intrinsic and extrinsic NLP tasks using public test sets.