Clustering-Aware Negative Sampling for Unsupervised Sentence Representation
This work addresses a specific bottleneck in unsupervised sentence representation learning, offering an incremental improvement for NLP researchers and practitioners.
The paper tackled the problem of selecting appropriate negative examples in contrastive learning for unsupervised sentence representation, proposing ClusterNS to incorporate clustering for hard negatives and false negative recognition, resulting in favorable performance on semantic textual similarity tasks compared to baselines.
Contrastive learning has been widely studied in sentence representation learning. However, earlier works mainly focus on the construction of positive examples, while in-batch samples are often simply treated as negative examples. This approach overlooks the importance of selecting appropriate negative examples, potentially leading to a scarcity of hard negatives and the inclusion of false negatives. To address these issues, we propose ClusterNS (Clustering-aware Negative Sampling), a novel method that incorporates cluster information into contrastive learning for unsupervised sentence representation learning. We apply a modified K-means clustering algorithm to supply hard negatives and recognize in-batch false negatives during training, aiming to solve the two issues in one unified framework. Experiments on semantic textual similarity (STS) tasks demonstrate that our proposed ClusterNS compares favorably with baselines in unsupervised sentence representation learning. Our code has been made publicly available.