Neural Discourse Modeling of Conversations
This work addresses the challenge of maintaining coherence in multi-turn conversations for natural language processing applications, representing an incremental improvement over existing methods.
The authors tackled the problem of modeling long-range discourse in conversations using neural networks, finding that adding an RNN layer for discourse and providing more context improved output quality, with quantitative evidence of increased coherence and cohesion.
Deep neural networks have shown recent promise in many language-related tasks such as the modeling of conversations. We extend RNN-based sequence to sequence models to capture the long range discourse across many turns of conversation. We perform a sensitivity analysis on how much additional context affects performance, and provide quantitative and qualitative evidence that these models are able to capture discourse relationships across multiple utterances. Our results quantifies how adding an additional RNN layer for modeling discourse improves the quality of output utterances and providing more of the previous conversation as input also improves performance. By searching the generated outputs for specific discourse markers we show how neural discourse models can exhibit increased coherence and cohesion in conversations.