Robust Dialog State Tracking for Large Ontologies
This addresses the problem of accurate dialog state tracking for large-scale systems, though it is incremental with tailored improvements.
The paper tackled robust dialog state tracking for large ontologies in DSTC 4, achieving first place with F1-scores 9 and 7 percentage points higher than the runner-up at utterance and subdialog levels.
The Dialog State Tracking Challenge 4 (DSTC 4) differentiates itself from the previous three editions as follows: the number of slot-value pairs present in the ontology is much larger, no spoken language understanding output is given, and utterances are labeled at the subdialog level. This paper describes a novel dialog state tracking method designed to work robustly under these conditions, using elaborate string matching, coreference resolution tailored for dialogs and a few other improvements. The method can correctly identify many values that are not explicitly present in the utterance. On the final evaluation, our method came in first among 7 competing teams and 24 entries. The F1-score achieved by our method was 9 and 7 percentage points higher than that of the runner-up for the utterance-level evaluation and for the subdialog-level evaluation, respectively.