Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
This work addresses the need for cross-lingual inferences in NLP tasks, benefiting applications involving knowledge bases and texts across languages, but it appears incremental as it builds on existing joint representation learning methods.
The paper tackled the problem of joint representation learning for cross-lingual words and entities, which had not been well explored, by proposing a novel method that uses attentive distant supervision without parallel corpora, achieving significant results in tasks like word translation, entity relatedness, and cross-lingual entity linking.
Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings. In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities. It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts. Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases. We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and cross-lingual attention to further reduce noises. We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking. The results, both qualitatively and quantitatively, demonstrate the significance of our method.