One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations
This addresses a limitation in sentence embedding methods for NLP applications like information retrieval and text classification, but it is incremental as it builds on existing contrastive learning frameworks.
The paper tackles the problem of sentence embeddings struggling to capture implicit semantics by proposing DualCSE, which assigns two embeddings per sentence for explicit and implicit meanings, improving downstream task performance.
Sentence embedding methods have made remarkable progress, yet they still struggle to capture the implicit semantics within sentences. This can be attributed to the inherent limitations of conventional sentence embedding methods that assign only a single vector per sentence. To overcome this limitation, we propose DualCSE, a sentence embedding method that assigns two embeddings to each sentence: one representing the explicit semantics and the other representing the implicit semantics. These embeddings coexist in the shared space, enabling the selection of the desired semantics for specific purposes such as information retrieval and text classification. Experimental results demonstrate that DualCSE can effectively encode both explicit and implicit meanings and improve the performance of the downstream task.