Probing Multilingual BERT for Genetic and Typological Signals
This work addresses the problem of interpreting cross-lingual text representations for linguists and NLP researchers, offering incremental insights into typological interpretability.
The study probed multilingual BERT to detect genetic and typological language signals across 100 languages, finding that language distances derived from mBERT representations closely matched reference family trees and were best explained by phylogenetic factors, with a novel measure for diachronic meaning stability correlating significantly with linguistic rankings.
We probe the layers in multilingual BERT (mBERT) for phylogenetic and geographic language signals across 100 languages and compute language distances based on the mBERT representations. We 1) employ the language distances to infer and evaluate language trees, finding that they are close to the reference family tree in terms of quartet tree distance, 2) perform distance matrix regression analysis, finding that the language distances can be best explained by phylogenetic and worst by structural factors and 3) present a novel measure for measuring diachronic meaning stability (based on cross-lingual representation variability) which correlates significantly with published ranked lists based on linguistic approaches. Our results contribute to the nascent field of typological interpretability of cross-lingual text representations.