Towards End-to-End Integration of Dialog History for Improved Spoken Language Understanding
This work addresses the challenge of making dialog systems more compact and robust to ASR errors for improved SLU performance, representing an incremental advance by integrating history in a novel way.
The paper tackles the problem of integrating dialog history into spoken language understanding (SLU) without relying on text-based automatic speech recognition (ASR), proposing a hierarchical conversation model that directly uses speech-form history to make the system fully end-to-end. It achieves a 7.7% absolute F1 score improvement over a history-independent baseline on dialog action recognition, uses 48% fewer parameters than a cascaded baseline, and outperforms that baseline by 10% absolute F1 score when gold transcripts are unavailable.
Dialog history plays an important role in spoken language understanding (SLU) performance in a dialog system. For end-to-end (E2E) SLU, previous work has used dialog history in text form, which makes the model dependent on a cascaded automatic speech recognizer (ASR). This rescinds the benefits of an E2E system which is intended to be compact and robust to ASR errors. In this paper, we propose a hierarchical conversation model that is capable of directly using dialog history in speech form, making it fully E2E. We also distill semantic knowledge from the available gold conversation transcripts by jointly training a similar text-based conversation model with an explicit tying of acoustic and semantic embeddings. We also propose a novel technique that we call DropFrame to deal with the long training time incurred by adding dialog history in an E2E manner. On the HarperValleyBank dialog dataset, our E2E history integration outperforms a history independent baseline by 7.7% absolute F1 score on the task of dialog action recognition. Our model performs competitively with the state-of-the-art history based cascaded baseline, but uses 48% fewer parameters. In the absence of gold transcripts to fine-tune an ASR model, our model outperforms this baseline by a significant margin of 10% absolute F1 score.