Towards Detecting Inconsistencies in End-to-end Generated TODs
For developers of conversational AI systems, this work offers a method to automatically detect hallucinations in TODs, a critical issue for task completion, but the approach is incremental as it applies CSP to a known problem.
The paper addresses the problem of detecting inconsistencies in end-to-end generated task-oriented dialogues (TODs), where LLMs may hallucinate information not in the knowledge base. They propose a CSP-based method that models dialogues as constraint satisfaction problems, achieving high accuracy in detecting inconsistencies, though no specific numbers are provided.
Generative AI is profoundly transforming the core technologies behind conversational systems, shifting from component-based to end-to-end approaches. However, Large Language Models (LLMs) may still generate inconsistencies, a critical issue particularly in Task-Oriented Dialogues (TODs), where system responses must strictly adhere to information from a domain knowledge base (e.g., restaurants in a city). A single hallucination (e.g., suggesting a non-existent restaurant) can lead to severe task failures. We investigate a method for automatically detecting inconsistencies by conceptualizing TODs as a Constraint Satisfaction Problem (CSP), where variables represent dialogue segments referencing the conversational domain, and constraints among variables capture dialogue properties such as turn coherence and adherence to domain knowledge. We propose a pipeline that first identifies variables in a target dialogue and then applies a CSP solver to identify valid solutions. By comparing the target dialogue with valid variable assignments, we can detect inconsistencies and suggest minimal changes to ensure dialogue consistency. We demonstrate the high accuracy of the CSP-based approach in detecting inconsistencies, and provide a detailed analysis of our findings.