Deconstructing Word Embeddings
This addresses theoretical inconsistencies in word embeddings for NLP researchers, but it is incremental as it focuses on a new theoretical framework without empirical validation.
The paper identifies shortcomings in existing word embedding models, such as instability and distorted analogical reasoning, and proposes a new theoretical model called Derridian Embedding to evaluate them qualitatively.
A review of Word Embedding Models through a deconstructive approach reveals their several shortcomings and inconsistencies. These include instability of the vector representations, a distorted analogical reasoning, geometric incompatibility with linguistic features, and the inconsistencies in the corpus data. A new theoretical embedding model, Derridian Embedding, is proposed in this paper. Contemporary embedding models are evaluated qualitatively in terms of how adequate they are in relation to the capabilities of a Derridian Embedding.