Handling Ontology Gaps in Semantic Parsing
This addresses the issue of building trusted question-answering agents by preventing offensive or wrong responses in closed-ontology semantic parsing, though it is incremental as it builds on existing detection techniques.
The paper tackles the problem of neural semantic parsing models generating hallucinated outputs due to the closed-world assumption, proposing a Hallucination Simulation Framework to analyze and detect hallucinations, which improves F1-Scores by ~21% for ontology gaps, ~24% for out-of-domain utterances, and ~1% for error recognition.
The majority of Neural Semantic Parsing (NSP) models are developed with the assumption that there are no concepts outside the ones such models can represent with their target symbols (closed-world assumption). This assumption leads to generate hallucinated outputs rather than admitting their lack of knowledge. Hallucinations can lead to wrong or potentially offensive responses to users. Hence, a mechanism to prevent this behavior is crucial to build trusted NSP-based Question Answering agents. To that end, we propose the Hallucination Simulation Framework (HSF), a general setting for stimulating and analyzing NSP model hallucinations. The framework can be applied to any NSP task with a closed-ontology. Using the proposed framework and KQA Pro as the benchmark dataset, we assess state-of-the-art techniques for hallucination detection. We then present a novel hallucination detection strategy that exploits the computational graph of the NSP model to detect the NSP hallucinations in the presence of ontology gaps, out-of-domain utterances, and to recognize NSP errors, improving the F1-Score respectively by ~21, ~24% and ~1%. This is the first work in closed-ontology NSP that addresses the problem of recognizing ontology gaps. We release our code and checkpoints at https://github.com/amazon-science/handling-ontology-gaps-in-semantic-parsing.