Intuitive Contrasting Map for Antonym Embeddings
This addresses the challenge of semantic attribute extraction in natural language processing for researchers and practitioners, though it is incremental as it builds on existing embedding methods.
The paper tackles the problem of distinguishing synonyms and antonyms in word embeddings despite small cosine similarities, showing that a contrasting map can extract this geometric information to produce new embeddings that improve downstream task performance.
This paper shows that, modern word embeddings contain information that distinguishes synonyms and antonyms despite small cosine similarities between corresponding vectors. This information is encoded in the geometry of the embeddings and could be extracted with a straight-forward and intuitive manifold learning procedure or a contrasting map. Such a map is trained on a small labeled subset of the data and can produce new embeddings that explicitly highlight specific semantic attributes of the word. The new embeddings produced by the map are shown to improve the performance on downstream tasks.