More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking
This addresses robustness issues in task-oriented dialogue agents for improved real-world deployment, though it is incremental as it builds on existing schema-guided paradigms.
The paper tackles the problem of schema-guided dialogue state tracking models being sensitive to schema writing style by proposing a framework that generates synthetic schemas using tree-based ranking to optimize lexical diversity and semantic faithfulness. The result shows marked improvements in average joint goal accuracy and schema sensitivity on the SGD-X benchmark when augmenting training data with these prompts.
The schema-guided paradigm overcomes scalability issues inherent in building task-oriented dialogue (TOD) agents with static ontologies. Instead of operating on dialogue context alone, agents have access to hierarchical schemas containing task-relevant natural language descriptions. Fine-tuned language models excel at schema-guided dialogue state tracking (DST) but are sensitive to the writing style of the schemas. We explore methods for improving the robustness of DST models. We propose a framework for generating synthetic schemas which uses tree-based ranking to jointly optimise lexical diversity and semantic faithfulness. The generalisation of strong baselines is improved when augmenting their training data with prompts generated by our framework, as demonstrated by marked improvements in average joint goal accuracy (JGA) and schema sensitivity (SS) on the SGD-X benchmark.