The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures
This work addresses the challenge of efficiently estimating cross-lingual transferability for NLP practitioners, though it is incremental as it builds on existing isomorphism concepts.
The paper tackled the problem of predicting cross-lingual NLP task performance by analyzing monolingual embedding space similarity, showing that spectral-based isomorphism measures strongly correlate with performance across tasks like bilingual lexicon induction and machine translation, and outperform previous measures while being more efficient.
Performance in cross-lingual NLP tasks is impacted by the (dis)similarity of languages at hand: e.g., previous work has suggested there is a connection between the expected success of bilingual lexicon induction (BLI) and the assumption of (approximate) isomorphism between monolingual embedding spaces. In this work we present a large-scale study focused on the correlations between monolingual embedding space similarity and task performance, covering thousands of language pairs and four different tasks: BLI, parsing, POS tagging and MT. We hypothesize that statistics of the spectrum of each monolingual embedding space indicate how well they can be aligned. We then introduce several isomorphism measures between two embedding spaces, based on the relevant statistics of their individual spectra. We empirically show that 1) language similarity scores derived from such spectral isomorphism measures are strongly associated with performance observed in different cross-lingual tasks, and 2) our spectral-based measures consistently outperform previous standard isomorphism measures, while being computationally more tractable and easier to interpret. Finally, our measures capture complementary information to typologically driven language distance measures, and the combination of measures from the two families yields even higher task performance correlations.