ESCL: Equivariant Self-Contrastive Learning for Sentence Representations
This work addresses the challenge of enhancing semantic sentence representations for natural language processing tasks, offering an incremental improvement over existing contrastive learning methods.
The paper tackles the problem of improving sentence representations by addressing the neglect of harmful sensitive transformations in contrastive learning, proposing an Equivariant Self-Contrastive Learning method that achieves better results on semantic textual similarity tasks with fewer parameters.
Previous contrastive learning methods for sentence representations often focus on insensitive transformations to produce positive pairs, but neglect the role of sensitive transformations that are harmful to semantic representations. Therefore, we propose an Equivariant Self-Contrastive Learning (ESCL) method to make full use of sensitive transformations, which encourages the learned representations to be sensitive to certain types of transformations with an additional equivariant learning task. Meanwhile, in order to improve practicability and generality, ESCL simplifies the implementations of traditional equivariant contrastive methods to share model parameters from the perspective of multi-task learning. We evaluate our ESCL on semantic textual similarity tasks. The proposed method achieves better results while using fewer learning parameters compared to previous methods.