KARLA: Knowledge-base Augmented Retrieval for Language Models
For LLM users and developers, KARLA provides a way to keep factual knowledge current and traceable without retraining, but the approach is incremental as it builds on existing retrieval-augmented generation paradigms.
KARLA enables LLMs to retrieve factual knowledge from a knowledge base during generation, allowing updates without retraining and improving factual accuracy. Experiments show improved factual grounding in short and long-form generation, with KB edits replacing parameter updates.
We propose a new method that allows an LLM to automatically pull in factual knowledge from a knowledge base during token generation. This means that (1)~factual knowledge in the LLM output can be updated without retraining the LLM, (2)~facts in the LLM output can be traced to the knowledge base for transparency and explainability, and (3)~smaller models can achieve the same factual accuracy as larger models. Our core idea is to train the model to produce special tokens that trigger a query to the knowledge base. Our experiments show that our method improves factual grounding in both short and long-form generation, and allows factual revisions to take effect through KB edits rather than parameter updates.