Wasserstein-Fisher-Rao Document Distance
This addresses a robustness issue in document distance measurement for natural language processing, though it is incremental as it builds on existing optimal transport methods.
The authors tackled the problem of robustly measuring semantic similarity between documents of different lengths by applying the Wasserstein-Fisher-Rao metric from unbalanced optimal transport theory, resulting in clear improvements over Word Mover's Distance in KNN classification across eight datasets.
As a fundamental problem of natural language processing, it is important to measure the distance between different documents. Among the existing methods, the Word Mover's Distance (WMD) has shown remarkable success in document semantic matching for its clear physical insight as a parameter-free model. However, WMD is essentially based on the classical Wasserstein metric, thus it often fails to robustly represent the semantic similarity between texts of different lengths. In this paper, we apply the newly developed Wasserstein-Fisher-Rao (WFR) metric from unbalanced optimal transport theory to measure the distance between different documents. The proposed WFR document distance maintains the great interpretability and simplicity as WMD. We demonstrate that the WFR document distance has significant advantages when comparing the texts of different lengths. In addition, an accelerated Sinkhorn based algorithm with GPU implementation has been developed for the fast computation of WFR distances. The KNN classification results on eight datasets have shown its clear improvement over WMD.