A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition
This addresses the challenge of obtaining reliable labels from crowdsourcing for NER, which is incremental as it builds on existing partial label learning approaches.
The paper tackles the problem of noisy labels in crowd-annotated named entity recognition by proposing a confidence-based partial label learning method that integrates prior and posterior confidences, showing effective performance improvements on real-world and synthetic datasets.
Existing models for named entity recognition (NER) are mainly based on large-scale labeled datasets, which always obtain using crowdsourcing. However, it is hard to obtain a unified and correct label via majority voting from multiple annotators for NER due to the large labeling space and complexity of this task. To address this problem, we aim to utilize the original multi-annotator labels directly. Particularly, we propose a Confidence-based Partial Label Learning (CPLL) method to integrate the prior confidence (given by annotators) and posterior confidences (learned by models) for crowd-annotated NER. This model learns a token- and content-dependent confidence via an Expectation-Maximization (EM) algorithm by minimizing empirical risk. The true posterior estimator and confidence estimator perform iteratively to update the true posterior and confidence respectively. We conduct extensive experimental results on both real-world and synthetic datasets, which show that our model can improve performance effectively compared with strong baselines.