Predictive Multiplicity of Knowledge Graph Embeddings in Link Prediction
This addresses a risk for high-stake applications using knowledge graph embeddings, but it is incremental as it applies known social choice methods to a new problem in this domain.
The paper tackles the problem of predictive multiplicity in knowledge graph embeddings for link prediction, where multiple models perform similarly but give conflicting predictions, and finds that 8% to 39% of testing queries exhibit such conflicts. It addresses this by using voting methods from social choice theory, reducing conflicts by 66% to 78% in experiments.
Knowledge graph embedding (KGE) models are often used to predict missing links for knowledge graphs (KGs). However, multiple KG embeddings can perform almost equally well for link prediction yet give conflicting predictions for unseen queries. This phenomenon is termed \textit{predictive multiplicity} in the literature. It poses substantial risks for KGE-based applications in high-stake domains but has been overlooked in KGE research. We define predictive multiplicity in link prediction, introduce evaluation metrics and measure predictive multiplicity for representative KGE methods on commonly used benchmark datasets. Our empirical study reveals significant predictive multiplicity in link prediction, with $8\%$ to $39\%$ testing queries exhibiting conflicting predictions. We address this issue by leveraging voting methods from social choice theory, significantly mitigating conflicts by $66\%$ to $78\%$ in our experiments.