Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction
This work addresses event extraction for natural language processing applications, offering a robust solution that is incremental in combining existing techniques.
The paper tackles the problem of event extraction from text by addressing the trade-off between precision in discriminative models and recall in generative LLMs, proposing a hybrid approach that combines a Self Mixture of Agents with a sequence tagger to improve prediction quality, and it demonstrates outperforming state-of-the-art methods on three benchmark datasets.
Event Extraction (EE) involves automatically identifying and extracting structured information about events from unstructured text, including triggers, event types, and arguments. Traditional discriminative models demonstrate high precision but often exhibit limited recall, particularly for nuanced or infrequent events. Conversely, generative approaches leveraging Large Language Models (LLMs) provide higher semantic flexibility and recall but suffer from hallucinations and inconsistent predictions. To address these challenges, we propose Agreement-based Reflective Inference System (ARIS), a hybrid approach combining a Self Mixture of Agents with a discriminative sequence tagger. ARIS explicitly leverages structured model consensus, confidence-based filtering, and an LLM reflective inference module to reliably resolve ambiguities and enhance overall event prediction quality. We further investigate decomposed instruction fine-tuning for enhanced LLM event extraction understanding. Experiments demonstrate our approach outperforms existing state-of-the-art event extraction methods across three benchmark datasets.