Beyond Completion: A Foundation Model for General Knowledge Graph Reasoning
This addresses the problem of extending knowledge graph reasoning beyond completion tasks to more challenging out-of-KG applications for NLP and AI researchers, representing a novel method rather than an incremental improvement.
The paper tackles the limitation of existing knowledge graph foundation models that focus primarily on structural aspects and in-KG tasks by introducing MERRY, a foundation model for general knowledge graph reasoning that integrates both structural and textual information. It demonstrates strong performance, outperforming baselines on 28 datasets across in-KG and out-of-KG tasks like KG question answering.
In natural language processing (NLP) and computer vision (CV), the successful application of foundation models across diverse tasks has demonstrated their remarkable potential. However, despite the rich structural and textual information embedded in knowledge graphs (KGs), existing research of foundation model for KG has primarily focused on their structural aspects, with most efforts restricted to in-KG tasks (e.g., knowledge graph completion, KGC). This limitation has hindered progress in addressing more challenging out-of-KG tasks. In this paper, we introduce MERRY, a foundation model for general knowledge graph reasoning, and investigate its performance across two task categories: in-KG reasoning tasks (e.g., KGC) and out-of-KG tasks (e.g., KG question answering, KGQA). We not only utilize the structural information, but also the textual information in KGs. Specifically, we propose a multi-perspective Conditional Message Passing (CMP) encoding architecture to bridge the gap between textual and structural modalities, enabling their seamless integration. Additionally, we introduce a dynamic residual fusion module to selectively retain relevant textual information and a flexible edge scoring mechanism to adapt to diverse downstream tasks. Comprehensive evaluations on 28 datasets demonstrate that MERRY outperforms existing baselines in most scenarios, showcasing strong reasoning capabilities within KGs and excellent generalization to out-of-KG tasks such as KGQA.