A Simple and Plug-and-play Method for Unsupervised Sentence Representation Enhancement
This addresses the need for better semantic matching in retrieval applications, but it is incremental as it builds on existing pre-trained models.
The paper tackles the problem of improving unsupervised sentence embeddings by introducing RepAL, a simple post-processing method that de-emphasizes redundant information, and shows it enhances performance without training, achieving competitive results on benchmarks like STS and transfer tasks.
Generating proper embedding of sentences through an unsupervised way is beneficial to semantic matching and retrieval problems in real-world scenarios. This paper presents Representation ALchemy (RepAL), an extremely simple post-processing method that enhances sentence representations. The basic idea in RepAL is to de-emphasize redundant information of sentence embedding generated by pre-trained models. Through comprehensive experiments, we show that RepAL is free of training and is a plug-and-play method that can be combined with most existing unsupervised sentence learning models. We also conducted in-depth analysis to understand RepAL.