M-MRE: Extending the Mutual Reinforcement Effect to Multimodal Information Extraction
This work addresses the applicability of MRE to visual and multimodal domains for researchers in information extraction and model interpretability, representing an incremental extension from textual to multimodal settings.
The authors tackled the problem of extending the Mutual Reinforcement Effect (MRE) to multimodal information extraction, introducing the M-MRE task and a corresponding dataset, and demonstrated that MRE facilitates mutual gains across three interrelated tasks in a multimodal scenario.
Mutual Reinforcement Effect (MRE) is an emerging subfield at the intersection of information extraction and model interpretability. MRE aims to leverage the mutual understanding between tasks of different granularities, enhancing the performance of both coarse-grained and fine-grained tasks through joint modeling. While MRE has been explored and validated in the textual domain, its applicability to visual and multimodal domains remains unexplored. In this work, we extend MRE to the multimodal information extraction domain for the first time. Specifically, we introduce a new task: Multimodal Mutual Reinforcement Effect (M-MRE), and construct a corresponding dataset to support this task. To address the challenges posed by M-MRE, we further propose a Prompt Format Adapter (PFA) that is fully compatible with various Large Vision-Language Models (LVLMs). Experimental results demonstrate that MRE can also be observed in the M-MRE task, a multimodal text-image understanding scenario. This provides strong evidence that MRE facilitates mutual gains across three interrelated tasks, confirming its generalizability beyond the textual domain.