In Search for Linear Relations in Sentence Embedding Spaces
This is an incremental study analyzing the properties of existing sentence embeddings for NLP researchers.
The authors investigated how small textual changes in sentences affect their vector representations in popular sentence embedding models, finding that vector differences in some embeddings reflect these alterations.
We present an introductory investigation into continuous-space vector representations of sentences. We acquire pairs of very similar sentences differing only by a small alterations (such as change of a noun, adding an adjective, noun or punctuation) from datasets for natural language inference using a simple pattern method. We look into how such a small change within the sentence text affects its representation in the continuous space and how such alterations are reflected by some of the popular sentence embedding models. We found that vector differences of some embeddings actually reflect small changes within a sentence.