Improving Semantic Parsing for Task Oriented Dialog
This work addresses semantic parsing errors in task-oriented dialog systems, offering incremental improvements over existing methods.
The paper tackled the problem of semantic parsing for task-oriented dialog by proposing three improvements: contextualized embeddings, ensembling, and pairwise re-ranking, which together achieved a 6.4% better exact match accuracy and 33% error reduction compared to the state-of-the-art on the TOP dataset.
Semantic parsing using hierarchical representations has recently been proposed for task oriented dialog with promising results [Gupta et al 2018]. In this paper, we present three different improvements to the model: contextualized embeddings, ensembling, and pairwise re-ranking based on a language model. We taxonomize the errors possible for the hierarchical representation, such as wrong top intent, missing spans or split spans, and show that the three approaches correct different kinds of errors. The best model combines the three techniques and gives 6.4% better exact match accuracy than the state-of-the-art, with an error reduction of 33%, resulting in a new state-of-the-art result on the Task Oriented Parsing (TOP) dataset.