TNCSE: Tensor's Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings
This work addresses the problem of improving unsupervised sentence embeddings for natural language processing tasks, representing an incremental advance by focusing on norm constraints in addition to orientation.
The paper tackled the problem of unsupervised sentence embedding representation by proposing a new training objective that constrains the module length features between positive samples, combined with ensemble learning in the TNCSE framework. The results show that TNCSE achieves state-of-the-art performance on seven semantic text similarity tasks and outperforms baselines in zero-shot evaluations.
Unsupervised sentence embedding representation has become a hot research topic in natural language processing. As a tensor, sentence embedding has two critical properties: direction and norm. Existing works have been limited to constraining only the orientation of the samples' representations while ignoring the features of their module lengths. To address this issue, we propose a new training objective that optimizes the training of unsupervised contrastive learning by constraining the module length features between positive samples. We combine the training objective of Tensor's Norm Constraints with ensemble learning to propose a new Sentence Embedding representation framework, TNCSE. We evaluate seven semantic text similarity tasks, and the results show that TNCSE and derived models are the current state-of-the-art approach; in addition, we conduct extensive zero-shot evaluations, and the results show that TNCSE outperforms other baselines.