Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning
This work addresses the problem of predicting future facts in temporal knowledge graphs for researchers and practitioners, showing that LLMs can match specialized methods without prior knowledge, though it is incremental as it applies existing LLM techniques to a new task.
The paper tackles temporal knowledge graph forecasting by applying large language models with in-context learning, achieving performance on par with state-of-the-art specialized models without fine-tuning or explicit structural modules, with minimal impact from hiding semantic information (e.g., ±0.4% Hit@1).
Temporal knowledge graph (TKG) forecasting benchmarks challenge models to predict future facts using knowledge of past facts. In this paper, we apply large language models (LLMs) to these benchmarks using in-context learning (ICL). We investigate whether and to what extent LLMs can be used for TKG forecasting, especially without any fine-tuning or explicit modules for capturing structural and temporal information. For our experiments, we present a framework that converts relevant historical facts into prompts and generates ranked predictions using token probabilities. Surprisingly, we observe that LLMs, out-of-the-box, perform on par with state-of-the-art TKG models carefully designed and trained for TKG forecasting. Our extensive evaluation presents performances across several models and datasets with different characteristics, compares alternative heuristics for preparing contextual information, and contrasts to prominent TKG methods and simple frequency and recency baselines. We also discover that using numerical indices instead of entity/relation names, i.e., hiding semantic information, does not significantly affect the performance ($\pm$0.4\% Hit@1). This shows that prior semantic knowledge is unnecessary; instead, LLMs can leverage the existing patterns in the context to achieve such performance. Our analysis also reveals that ICL enables LLMs to learn irregular patterns from the historical context, going beyond simple predictions based on common or recent information.