Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs
This work challenges the assumption that domain-specific fine-tuning improves factuality, highlighting a critical problem for high-stakes scientific communication.
The paper introduces SciFactCheck, a benchmark of 2,500 prompts across five scientific domains, and finds that scientifically fine-tuned LLMs hallucinate more than their general-purpose bases across all hallucination types and domains, with fine-tuned models being less confident yet more assertive.
Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses significant risks in this high stakes use-case. Prior hallucination evaluation work remains largely restricted to the biomedical domain, treats hallucination as a binary task, and has not examined the growing family of scientifically fine-tuned LLMs. We address these gaps with SciFactCheck, a benchmark of 2,500 prompts across five scientific domains, paired with a modular evaluation framework targeting three factuality hallucination types: unverifiability, overclaim, and attribution. Using a controlled minimal-pairing design, we evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base. Our results indicate that 1. Scientifically fine-tuned models exhibit degraded factual reliability across all hallucination types and scientific domains, and 2. Fine-tuned models are internally less confident yet linguistically more assertive. A human pilot study further reveals that current fact-checking tools show only modest agreement with expert judgments on scientific content, and that defining scientifically check-worthy claims remains contested even among human annotators. Our findings fundamentally challenge current methods of domain-specific fine-tuning for factuality and call for developing improved verification infrastructure for scientific content.