Counterfactual Reasoning with Knowledge Graph Embeddings
This work addresses the challenge of evaluating and adapting knowledge graph embeddings for hypothetical scenarios, which is incremental in connecting two distinct areas for AI researchers.
The paper tackles the problem of linking knowledge graph completion and counterfactual reasoning by introducing the CFKGR task, where results show that KGEs adapted with COULDD detect plausible counterfactual changes following learned patterns but struggle with changes that do not, while ChatGPT outperforms in detection but has poor knowledge retention.
Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories. In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules. We create corresponding benchmark datasets, which contain diverse hypothetical scenarios with plausible changes to the original knowledge graph and facts that should be retained. We develop COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluate it on our benchmark. Our results indicate that KGEs learn patterns in the graph without explicit training. We further observe that KGEs adapted with COULDD solidly detect plausible counterfactual changes to the graph that follow these patterns. An evaluation on human-annotated data reveals that KGEs adapted with COULDD are mostly unable to recognize changes to the graph that do not follow learned inference rules. In contrast, ChatGPT mostly outperforms KGEs in detecting plausible changes to the graph but has poor knowledge retention. In summary, CFKGR connects two previously distinct areas, namely KG completion and counterfactual reasoning.