Fine-Tuned Large Language Models for Logical Translation: Reducing Hallucinations with Lang2Logic
This addresses the need for precise automated reasoning in software systems, though it appears incremental as it builds on existing LLM and NLP techniques.
The paper tackles the problem of hallucinations in large language models (LLMs) when translating natural language to formal logic, introducing a framework that combines fine-tuned LLMs with symbolic methods to reduce errors and generate reliable Conjunctive Normal Form (CNF) outputs.
Recent advances in natural language processing (NLP), particularly large language models (LLMs), have motivated the automatic translation of natural language statements into formal logic without human intervention. This enables automated reasoning and facilitates debugging, finding loop invariants, and adhering to specifications in software systems. However, hallucinations-incorrect outputs generated by LLMs are challenging, particularly for logical translation tasks requiring precision. This work introduces a novel framework that inputs English sentences, converts them into logical expressions, and then translates them into Conjunctive Normal Form (CNF) for satisfiability solving. It employs classical NLP techniques with self-defined grammar, symbolic computation libraries, and a fine-tuned language model to reduce hallucinations. In the early experiments, we observed that the fine-tuned model, trained on different grammar settings, could intentionally correct the same types of hallucinations made by the original model. Thus, it provides reliable CNF generation.