Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents
This addresses noisy label issues in distant supervision for entity and relation extraction, offering an incremental improvement over existing methods.
The paper tackled the problem of shifted label distribution in distant supervision for entity and relation extraction, caused by noisy training data and pipelined extraction, by proposing a joint extraction approach using cooperative multiagents to re-label noisy instances with confidence scores, resulting in significant outperformance over state-of-the-art methods on two real-world datasets.
Distant supervision based methods for entity and relation extraction have received increasing popularity due to the fact that these methods require light human annotation efforts. In this paper, we consider the problem of \textit{shifted label distribution}, which is caused by the inconsistency between the noisy-labeled training set subject to external knowledge graph and the human-annotated test set, and exacerbated by the pipelined entity-then-relation extraction manner with noise propagation. We propose a joint extraction approach to address this problem by re-labeling noisy instances with a group of cooperative multiagents. To handle noisy instances in a fine-grained manner, each agent in the cooperative group evaluates the instance by calculating a continuous confidence score from its own perspective; To leverage the correlations between these two extraction tasks, a confidence consensus module is designed to gather the wisdom of all agents and re-distribute the noisy training set with confidence-scored labels. Further, the confidences are used to adjust the training losses of extractors. Experimental results on two real-world datasets verify the benefits of re-labeling noisy instance, and show that the proposed model significantly outperforms the state-of-the-art entity and relation extraction methods.