Certainty in Uncertainty: Reasoning over Uncertain Knowledge Graphs with Statistical Guarantees
This addresses reliability issues in high-stakes applications by providing uncertainty estimates for uncertain knowledge graph predictions, though it is incremental as it builds on conformal prediction.
The paper tackled the problem of uncertain knowledge graph embedding methods lacking predictive uncertainty quantification, proposing UnKGCP to generate prediction intervals with statistical guarantees, and empirically showed these intervals are sharp and effective across benchmarks.
Uncertain knowledge graph embedding (UnKGE) methods learn vector representations that capture both structural and uncertainty information to predict scores of unseen triples. However, existing methods produce only point estimates, without quantifying predictive uncertainty-limiting their reliability in high-stakes applications where understanding confidence in predictions is crucial. To address this limitation, we propose \textsc{UnKGCP}, a framework that generates prediction intervals guaranteed to contain the true score with a user-specified level of confidence. The length of the intervals reflects the model's predictive uncertainty. \textsc{UnKGCP} builds on the conformal prediction framework but introduces a novel nonconformity measure tailored to UnKGE methods and an efficient procedure for interval construction. We provide theoretical guarantees for the intervals and empirically verify these guarantees. Extensive experiments on standard benchmarks across diverse UnKGE methods further demonstrate that the intervals are sharp and effectively capture predictive uncertainty.