Context-Aware Interaction Network for Question Matching
This work addresses question matching for natural language processing applications, presenting an incremental improvement over existing cross-attention methods.
The paper tackles the problem of text matching for question pairs by addressing the limitation of regular cross-attention mechanisms that neglect contextual information, proposing a context-aware interaction network (COIN) that integrates contextual details and uses gate fusion layers. Experiments on two question matching datasets show the model's effectiveness, though no concrete performance numbers are provided in the abstract.
Impressive milestones have been achieved in text matching by adopting a cross-attention mechanism to capture pertinent semantic connections between two sentence representations. However, regular cross-attention focuses on word-level links between the two input sequences, neglecting the importance of contextual information. We propose a context-aware interaction network (COIN) to properly align two sequences and infer their semantic relationship. Specifically, each interaction block includes (1) a context-aware cross-attention mechanism to effectively integrate contextual information when aligning two sequences, and (2) a gate fusion layer to flexibly interpolate aligned representations. We apply multiple stacked interaction blocks to produce alignments at different levels and gradually refine the attention results. Experiments on two question matching datasets and detailed analyses demonstrate the effectiveness of our model.