LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning
This addresses the deployment challenge of temporal knowledge graph reasoning systems for resource-constrained applications, though it appears incremental as it builds on existing distillation techniques.
The paper tackles the problem of computationally heavy temporal knowledge graph reasoning models by proposing an LLM-assisted distillation framework that incorporates a large language model as an auxiliary instructor to provide enriched supervision. The approach consistently improves link prediction performance over strong distillation baselines on multiple TKG benchmarks while maintaining a compact and efficient student model.
Temporal knowledge graphs (TKGs) support reasoning over time-evolving facts, yet state-of-the-art models are often computationally heavy and costly to deploy. Existing compression and distillation techniques are largely designed for static graphs; directly applying them to temporal settings may overlook time-dependent interactions and lead to performance degradation. We propose an LLM-assisted distillation framework specifically designed for temporal knowledge graph reasoning. Beyond a conventional high-capacity temporal teacher, we incorporate a large language model as an auxiliary instructor to provide enriched supervision. The LLM supplies broad background knowledge and temporally informed signals, enabling a lightweight student to better model event dynamics without increasing inference-time complexity. Training is conducted by jointly optimizing supervised and distillation objectives, using a staged alignment strategy to progressively integrate guidance from both teachers. Extensive experiments on multiple public TKG benchmarks with diverse backbone architectures demonstrate that the proposed approach consistently improves link prediction performance over strong distillation baselines, while maintaining a compact and efficient student model. The results highlight the potential of large language models as effective teachers for transferring temporal reasoning capability to resource-efficient TKG systems.