FLAME: Empowering Frozen LLMs for Knowledge Graph Completion
This addresses the trade-off between cost and performance in using LLMs for knowledge graph completion, offering a more efficient solution for researchers and practitioners in AI and data management.
The paper tackles knowledge graph completion by proposing FLAME, a framework that uses frozen LLMs to extract hidden states for training efficient classifiers, achieving a 47% improvement over non-fine-tuned baselines and matching fine-tuned performance with 188x memory efficiency and 26.11x speedup.
Traditional knowledge graph completion (KGC) methods rely solely on structural information and struggle with sparsity, while Large Language Models (LLMs) address these limitations through rich world knowledge and strong context modeling. Fine-tuning LLMs is effective but costly, while non-fine-tuned LLMs are efficient but suboptimal. To address this trade-off, we propose \textbf{FLAME}, a framework that extracts context-aware hidden states from intermediate layers of frozen LLMs to train data-efficient KGC classifiers. We bridge LLM-KG semantic gaps via subgraph-based entity descriptions and employ sliced mutual information (SMI) to quantify task-relevant information in representations. Experiments demonstrate that FLAME achieves 47\% improvement over non-fine-tuned LLM baselines and, to our knowledge, is the first to achieve fine-tuned performance with $188\times$ memory efficiency and $26.11\times$ speedup.