Structural-Aware Sentence Similarity with Recursive Optimal Transport
This work addresses a practical problem in natural language processing for applications requiring efficient and accurate sentence similarity measures, but it is incremental as it builds on existing optimal transport methods.
The paper tackled the problem of measuring sentence similarity by addressing the inability of existing light-weighted models to detect structural differences, such as word order, and introduced Recursive Optimal Transport (ROT) and Recursive Optimal Similarity (ROTS) to incorporate structural information with low time complexity, achieving clear advantages over all weakly supervised approaches in experiments across 20 STS datasets.
Measuring sentence similarity is a classic topic in natural language processing. Light-weighted similarities are still of particular practical significance even when deep learning models have succeeded in many other tasks. Some light-weighted similarities with more theoretical insights have been demonstrated to be even stronger than supervised deep learning approaches. However, the successful light-weighted models such as Word Mover's Distance [Kusner et al., 2015] or Smooth Inverse Frequency [Arora et al., 2017] failed to detect the difference from the structure of sentences, i.e. order of words. To address this issue, we present Recursive Optimal Transport (ROT) framework to incorporate the structural information with the classic OT. Moreover, we further develop Recursive Optimal Similarity (ROTS) for sentences with the valuable semantic insights from the connections between cosine similarity of weighted average of word vectors and optimal transport. ROTS is structural-aware and with low time complexity compared to optimal transport. Our experiments over 20 sentence textural similarity (STS) datasets show the clear advantage of ROTS over all weakly supervised approaches. Detailed ablation study demonstrate the effectiveness of ROT and the semantic insights.