Improve Retrieval-based Dialogue System via Syntax-Informed Attention
This work addresses the problem of improving response selection in dialogue systems for users by introducing a novel attention mechanism, though it is incremental as it builds on existing syntax-based methods.
The paper tackled the challenge of multi-turn response selection in retrieval-based dialogue systems by proposing Syntax-Informed Attention (SIA), which incorporates both intra- and inter-sentence syntax information, and demonstrated general superiority on three benchmarks.
Multi-turn response selection is a challenging task due to its high demands on efficient extraction of the matching features from abundant information provided by context utterances. Since incorporating syntactic information like dependency structures into neural models can promote a better understanding of the sentences, such a method has been widely used in NLP tasks. Though syntactic information helps models achieved pleasing results, its application in retrieval-based dialogue systems has not been fully explored. Meanwhile, previous works focus on intra-sentence syntax alone, which is far from satisfactory for the task of multi-turn response where dialogues usually contain multiple sentences. To this end, we propose SIA, Syntax-Informed Attention, considering both intra- and inter-sentence syntax information. While the former restricts attention scope to only between tokens and corresponding dependents in the syntax tree, the latter allows attention in cross-utterance pairs for those syntactically important tokens. We evaluate our method on three widely used benchmarks and experimental results demonstrate the general superiority of our method on dialogue response selection.