History repeats Itself: A Baseline for Temporal Knowledge Graph Forecasting
This work addresses a critical evaluation gap for researchers in TKG forecasting, highlighting that current state-of-the-art methods may not be as effective as assumed, making it an incremental but important contribution.
The paper tackles the problem of evaluating Temporal Knowledge Graph (TKG) forecasting models by proposing a simple baseline that predicts recurring facts, revealing that it outperforms 11 existing methods on three out of five datasets, ranking first or third.
Temporal Knowledge Graph (TKG) Forecasting aims at predicting links in Knowledge Graphs for future timesteps based on a history of Knowledge Graphs. To this day, standardized evaluation protocols and rigorous comparison across TKG models are available, but the importance of simple baselines is often neglected in the evaluation, which prevents researchers from discerning actual and fictitious progress. We propose to close this gap by designing an intuitive baseline for TKG Forecasting based on predicting recurring facts. Compared to most TKG models, it requires little hyperparameter tuning and no iterative training. Further, it can help to identify failure modes in existing approaches. The empirical findings are quite unexpected: compared to 11 methods on five datasets, our baseline ranks first or third in three of them, painting a radically different picture of the predictive quality of the state of the art.