Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field
This work addresses the challenge of integrating cross-instance interactions in joint Information Extraction, which is incremental as it builds on prior methods by extending to higher-order dependencies.
The authors tackled the problem of modeling instance interactions in joint Information Extraction by introducing a high-order Conditional Random Field framework (CRFIE) that uses binary and ternary factors to capture interactions between pairs and triplets of instances, achieving consistent improvements on three IE tasks compared to baselines and prior work.
Prior works on joint Information Extraction (IE) typically model instance (e.g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding. We find that the previous models generally consider binary type dependency scoring of a pair of instances, and leverage local search such as beam search to approximate global solutions. To better integrate cross-instance interactions, in this work, we introduce a joint IE framework (CRFIE) that formulates joint IE as a high-order Conditional Random Field. Specifically, we design binary factors and ternary factors to directly model interactions between not only a pair of instances but also triplets. Then, these factors are utilized to jointly predict labels of all instances. To address the intractability problem of exact high-order inference, we incorporate a high-order neural decoder that is unfolded from a mean-field variational inference method, which achieves consistent learning and inference. The experimental results show that our approach achieves consistent improvements on three IE tasks compared with our baseline and prior work.