MEG: Medical Knowledge-Augmented Large Language Models for Question Answering
This work addresses the challenge of reliable reasoning in medical question answering for AI systems, though it is incremental as it builds on existing methods with a novel integration approach.
The paper tackled the problem of large language models struggling with nuanced concept relationships in specialized fields like medicine by introducing MEG, a parameter-efficient approach that incorporates knowledge graph embeddings, resulting in average accuracy improvements of +6.7% and +9.9% over specialized models on medical multiple-choice datasets.
Question answering is a natural language understanding task that involves reasoning over both explicit context, and unstated relevant domain knowledge. Despite the high cost of training, large language models (LLMs) -- the backbone of most modern question-answering systems -- still struggle to reliably capture the nuanced relationships between concepts that are crucial for reasoning in specialized fields like medicine. In this work, we present MEG, a parameter-efficient approach for medical knowledge-augmented LLMs. MEG uses a lightweight mapping network to incorporate knowledge graph embeddings into the LLM, enabling it to leverage external knowledge in a cost-effective way. We evaluate our method on four popular medical multiple-choice datasets and show that LLMs i) can effectively interpret knowledge graph embeddings and ii) gain significant advantages from the factual grounding these embeddings provide. MEG attains an average of +6.7% and +9.9% accuracy over specialized models like BioMistral-7B and MediTron-7B, respectively. Finally, we show that MEG's performance remains robust to the choice of graph encoder.