Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision
This work addresses noise in distant supervision for relation extraction, an incremental improvement for NLP researchers and practitioners.
The paper tackles the problem of noise in distant supervision during contrastive pre-training for relation extraction by proposing a weighted contrastive learning method that uses supervised data to estimate instance reliability and reduce noise effects, achieving advantages over state-of-the-art non-weighted baselines on three supervised datasets.
Contrastive pre-training on distant supervision has shown remarkable effectiveness in improving supervised relation extraction tasks. However, the existing methods ignore the intrinsic noise of distant supervision during the pre-training stage. In this paper, we propose a weighted contrastive learning method by leveraging the supervised data to estimate the reliability of pre-training instances and explicitly reduce the effect of noise. Experimental results on three supervised datasets demonstrate the advantages of our proposed weighted contrastive learning approach compared to two state-of-the-art non-weighted baselines.Our code and models are available at: https://github.com/YukinoWan/WCL