MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Training
This work addresses the problem of inconsistent hallucinations in LLMs for users relying on accurate and logical outputs, offering a generalizable solution without task-specific adaptation.
The paper tackles inconsistent hallucinations in large language models (LLMs) by proposing a novel framework that uses event-driven text-code cyclic training to transfer logical consistency from code to natural language, significantly reducing hallucinations across three LLMs and two task categories while maintaining overall performance.
Recent methodologies utilizing synthetic datasets have aimed to address inconsistent hallucinations in large language models (LLMs); however,these approaches are primarily tailored to specific tasks, limiting their generalizability. Inspired by the strong performance of code-trained models in logic-intensive domains, we propose a novel framework that leverages event-based text to generate corresponding code and employs cyclic training to transfer the logical consistency of code to natural language effectively. Our method significantly reduces inconsistent hallucinations across three leading LLMs and two categories of natural language tasks while maintaining overall performance. This framework effectively alleviates hallucinations without necessitating adaptation to downstream tasks, demonstrating generality and providing new perspectives to tackle the challenge of inconsistent hallucinations.