Conditional Generation and Snapshot Learning in Neural Dialogue Systems
This work addresses the challenge of model interpretability and performance in dialogue systems, but it is incremental as it builds on existing LSTM-based conditional language models.
The paper tackled the problem of improving neural dialogue systems by exploring different model architectures and conditioning methods, and introduced snapshot learning to enhance learning from sequential signals, resulting in consistent performance improvements across architectures.
Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential signals by applying a companion cross-entropy objective function to the conditioning vector. The experimental and analytical results demonstrate firstly that competition occurs between the conditioning vector and the LM, and the differing architectures provide different trade-offs between the two. Secondly, the discriminative power and transparency of the conditioning vector is key to providing both model interpretability and better performance. Thirdly, snapshot learning leads to consistent performance improvements independent of which architecture is used.