Identifying Domain Adjacent Instances for Semantic Parsers
This addresses a subtle failure mode in semantic parsing for NLP applications, but it is incremental as it builds on existing out-of-domain detection methods.
The paper tackled the problem of detecting domain-adjacent instances where semantic parsing fails due to schema limitations, even within the same domain, and proposed a sentence representation focusing on unexpected words, improving parser performance on in-domain and domain-adjacent cases.
When the semantics of a sentence are not representable in a semantic parser's output schema, parsing will inevitably fail. Detection of these instances is commonly treated as an out-of-domain classification problem. However, there is also a more subtle scenario in which the test data is drawn from the same domain. In addition to formalizing this problem of domain-adjacency, we present a comparison of various baselines that could be used to solve it. We also propose a new simple sentence representation that emphasizes words which are unexpected. This approach improves the performance of a downstream semantic parser run on in-domain and domain-adjacent instances.