EnCore: Fine-Grained Entity Typing by Pre-Training Entity Encoders on Coreference Chains
This work addresses the challenge of limited manual annotations in fine-grained entity typing for NLP applications, representing an incremental improvement over existing distant supervision methods.
The paper tackled the problem of noisy coreference links in fine-grained entity typing by pre-training entity encoders on coreference chains, using a trick of only considering links predicted by two different systems, resulting in improved state-of-the-art performance on benchmarks for fine-grained entity typing and entity extraction.
Entity typing is the task of assigning semantic types to the entities that are mentioned in a text. In the case of fine-grained entity typing (FET), a large set of candidate type labels is considered. Since obtaining sufficient amounts of manual annotations is then prohibitively expensive, FET models are typically trained using distant supervision. In this paper, we propose to improve on this process by pre-training an entity encoder such that embeddings of coreferring entities are more similar to each other than to the embeddings of other entities. The main problem with this strategy, which helps to explain why it has not previously been considered, is that predicted coreference links are often too noisy. We show that this problem can be addressed by using a simple trick: we only consider coreference links that are predicted by two different off-the-shelf systems. With this prudent use of coreference links, our pre-training strategy allows us to improve the state-of-the-art in benchmarks on fine-grained entity typing, as well as traditional entity extraction.