A Study on Building Efficient Zero-Shot Relation Extraction Models
This work addresses inefficiencies in zero-shot relation extraction for information retrieval applications, but it is incremental as it adapts existing methods rather than introducing new ones.
The study tackled the problem of adapting zero-shot relation extraction models to realistic scenarios by addressing unrealistic assumptions like lack of pre-computation and rejection mechanisms, showing that no existing model is fully robust but AlignRE performs best overall.
Zero-shot relation extraction aims to identify relations between entity mentions using textual descriptions of novel types (i.e., previously unseen) instead of labeled training examples. Previous works often rely on unrealistic assumptions: (1) pairs of mentions are often encoded directly in the input, which prevents offline pre-computation for large scale document database querying; (2) no rejection mechanism is introduced, biasing the evaluation when using these models in a retrieval scenario where some (and often most) inputs are irrelevant and must be ignored. In this work, we study the robustness of existing zero-shot relation extraction models when adapting them to a realistic extraction scenario. To this end, we introduce a typology of existing models, and propose several strategies to build single pass models and models with a rejection mechanism. We adapt several state-of-the-art tools, and compare them in this challenging setting, showing that no existing work is really robust to realistic assumptions, but overall AlignRE (Li et al., 2024) performs best along all criteria.