Collaborative Policy Learning for Open Knowledge Graph Reasoning
This addresses the challenge of incomplete knowledge graphs for AI systems requiring interpretable reasoning, though it is an incremental improvement over existing methods.
The paper tackles the problem of open knowledge graph reasoning, where models struggle with sparse graphs by proposing a collaborative reinforcement learning framework with two agents—a graph reasoner and a fact extractor—to dynamically enrich the graph and filter irrelevant facts, achieving improved performance on two public datasets.
In recent years, there has been a surge of interests in interpretable graph reasoning methods. However, these models often suffer from limited performance when working on sparse and incomplete graphs, due to the lack of evidential paths that can reach target entities. Here we study open knowledge graph reasoning---a task that aims to reason for missing facts over a graph augmented by a background text corpus. A key challenge of the task is to filter out "irrelevant" facts extracted from corpus, in order to maintain an effective search space during path inference. We propose a novel reinforcement learning framework to train two collaborative agents jointly, i.e., a multi-hop graph reasoner and a fact extractor. The fact extraction agent generates fact triples from corpora to enrich the graph on the fly; while the reasoning agent provides feedback to the fact extractor and guides it towards promoting facts that are helpful for the interpretable reasoning. Experiments on two public datasets demonstrate the effectiveness of the proposed approach. Source code and datasets used in this paper can be downloaded at https://github.com/shanzhenren/CPL