Evaluating Hallucinations in Domain-Adapted Large Language Models
For practitioners adapting LLMs to specialized domains, this work highlights the persistent challenge of hallucinations with fine-tuning-only methods, underscoring the need for more robust adaptation techniques.
The study evaluates hallucinations in Llama-2 fine-tuned on the Lamini dataset, finding that while the model performs well on tasks similar to its training data, it struggles with accurate reasoning and recall of new domain-specific information, leading to hallucinations. The model shows a tendency to over-generate, indicating limitations of fine-tuning-only approaches for domain adaptation.
This study investigates the phenomenon of hallucinations in domain-adapted Large Language Models (LLMs), focusing on the fine-tuning of the Llama-2 model with the Lamini dataset. Hallucinations, or the generation of nonsensical or unfaithful content by LLMs, pose a significant challenge, especially when these models are fine-tuned with domain-specific data. Our methodology involves a series of experiments testing memorization, recall, and reasoning capabilities of the fine-tuned LLM, comparing its performance on novel question-answer pairs and domain-specific information. We found that while the model shows proficiency in tasks similar to its training data, its capability to accurately reason about and recall new domain-specific information remains limited, leading to instances of hallucination. The model demonstrates a tendency to provide correct answers with extra information, suggesting an inclination toward over-generation. These results suggest important limitations of fine-tuning-only approaches for mitigating hallucinations when adapting LLMs to specialized domains and underscore the need for more robust methods in adapting LLMs to specialized domains. The study also provides insights into the varying performance of LLMs on different types of information, revealing a comparative weakness in handling domain-specific queries.