GLOSS: Generative Latent Optimization of Sentence Representations
This addresses the problem of learning sentence representations without compositional assumptions for NLP researchers, though it appears incremental as it builds on existing generative optimization techniques.
The paper tackles unsupervised sentence representation learning by proposing a non-compositional method based on Generative Latent Optimization, achieving a 5% relative improvement over uSIF and competitive performance to Sent2vec with 30 times less data.
We propose a method to learn unsupervised sentence representations in a non-compositional manner based on Generative Latent Optimization. Our approach does not impose any assumptions on how words are to be combined into a sentence representation. We discuss a simple Bag of Words model as well as a variant that models word positions. Both are trained to reconstruct the sentence based on a latent code and our model can be used to generate text. Experiments show large improvements over the related Paragraph Vectors. Compared to uSIF, we achieve a relative improvement of 5% when trained on the same data and our method performs competitively to Sent2vec while trained on 30 times less data.