CLLD: Contrastive Learning with Label Distance for Text Classification
This addresses a specific issue in text classification for NLP applications, but it is incremental as it builds on existing contrastive learning and pre-trained model frameworks.
The paper tackles the problem of semantic discrepancy between similar texts in pre-trained models for text classification, proposing CLLD to improve performance on hard-to-distinguish classes, with experiments showing enhanced results on benchmarks and internal datasets.
Existed pre-trained models have achieved state-of-the-art performance on various text classification tasks. These models have proven to be useful in learning universal language representations. However, the semantic discrepancy between similar texts cannot be effectively distinguished by advanced pre-trained models, which have a great influence on the performance of hard-to-distinguish classes. To address this problem, we propose a novel Contrastive Learning with Label Distance (CLLD) in this work. Inspired by recent advances in contrastive learning, we specifically design a classification method with label distance for learning contrastive classes. CLLD ensures the flexibility within the subtle differences that lead to different label assignments, and generates the distinct representations for each class having similarity simultaneously. Extensive experiments on public benchmarks and internal datasets demonstrate that our method improves the performance of pre-trained models on classification tasks. Importantly, our experiments suggest that the learned label distance relieve the adversarial nature of interclasses.