Large Language Model Enhanced Knowledge Representation Learning: A Survey
It surveys incremental advancements in KRL for applications using symbolic knowledge, focusing on integrating textual information to mitigate KG sparsity.
This survey addresses the problem of sparseness in Knowledge Graphs (KGs) by reviewing how Large Language Models (LLMs) enhance Knowledge Representation Learning (KRL) to improve effectiveness and generalization for downstream tasks, though no concrete numbers are provided.
Knowledge Representation Learning (KRL) is crucial for enabling applications of symbolic knowledge from Knowledge Graphs (KGs) to downstream tasks by projecting knowledge facts into vector spaces. Despite their effectiveness in modeling KG structural information, KRL methods are suffering from the sparseness of KGs. The rise of Large Language Models (LLMs) built on the Transformer architecture presents promising opportunities for enhancing KRL by incorporating textual information to address information sparsity in KGs. LLM-enhanced KRL methods, including three key approaches, encoder-based methods that leverage detailed contextual information, encoder-decoder-based methods that utilize a unified Seq2Seq model for comprehensive encoding and decoding, and decoder-based methods that utilize extensive knowledge from large corpora, have significantly advanced the effectiveness and generalization of KRL in addressing a wide range of downstream tasks. This work provides a broad overview of downstream tasks while simultaneously identifying emerging research directions in these evolving domains.