Graph Neural Prompting with Large Language Models
This addresses the limitation of LLMs in capturing precise knowledge for tasks like reasoning, offering a more efficient alternative to joint training for researchers and practitioners.
The authors tackled the problem of enhancing pre-trained large language models (LLMs) with grounded knowledge from knowledge graphs (KGs) without costly retraining, proposing Graph Neural Prompting (GNP) as a plug-and-play method that improved performance on commonsense and biomedical reasoning tasks across various LLM sizes and settings.
Large language models (LLMs) have shown remarkable generalization capability with exceptional performance in various language modeling tasks. However, they still exhibit inherent limitations in precisely capturing and returning grounded knowledge. While existing work has explored utilizing knowledge graphs (KGs) to enhance language modeling via joint training and customized model architectures, applying this to LLMs is problematic owing to their large number of parameters and high computational cost. Therefore, how to enhance pre-trained LLMs using grounded knowledge, e.g., retrieval-augmented generation, remains an open question. In this work, we propose Graph Neural Prompting (GNP), a novel plug-and-play method to assist pre-trained LLMs in learning beneficial knowledge from KGs. GNP encompasses various designs, including a standard graph neural network encoder, a cross-modality pooling module, a domain projector, and a self-supervised link prediction objective. Extensive experiments on multiple datasets demonstrate the superiority of GNP on both commonsense and biomedical reasoning tasks across different LLM sizes and settings. Code is available at https://github.com/meettyj/GNP.