Dialog Context Language Modeling with Recurrent Neural Networks
This work addresses the challenge of modeling dialog interactions for natural language processing applications, representing an incremental advancement over existing contextual language models.
The authors tackled the problem of incorporating dialog-level discourse information into language modeling by proposing RNN-based models that track speaker interactions, resulting in a 3.3% improvement in perplexity on the Switchboard Dialog Act Corpus compared to conventional single-turn models.
In this work, we propose contextual language models that incorporate dialog level discourse information into language modeling. Previous works on contextual language model treat preceding utterances as a sequence of inputs, without considering dialog interactions. We design recurrent neural network (RNN) based contextual language models that specially track the interactions between speakers in a dialog. Experiment results on Switchboard Dialog Act Corpus show that the proposed model outperforms conventional single turn based RNN language model by 3.3% on perplexity. The proposed models also demonstrate advantageous performance over other competitive contextual language models.