Word Representations via Gaussian Embedding
This work addresses the need for more expressive word representations in natural language processing, offering a novel approach that could enhance tasks like entailment modeling, though it is incremental in advancing embedding techniques.
The paper tackles the problem of representing words as point vectors by proposing density-based embeddings using Gaussian distributions, which better capture uncertainty and asymmetric relationships, and demonstrates improved performance on word embedding benchmarks.
Current work in lexical distributed representations maps each word to a point vector in low-dimensional space. Mapping instead to a density provides many interesting advantages, including better capturing uncertainty about a representation and its relationships, expressing asymmetries more naturally than dot product or cosine similarity, and enabling more expressive parameterization of decision boundaries. This paper advocates for density-based distributed embeddings and presents a method for learning representations in the space of Gaussian distributions. We compare performance on various word embedding benchmarks, investigate the ability of these embeddings to model entailment and other asymmetric relationships, and explore novel properties of the representation.