A Kernel Independence Test for Geographical Language Variation
This provides a more flexible tool for linguists and dialectologists to analyze geographical language variation without being limited by data type or model assumptions, though it is incremental in improving existing statistical tests.
The authors tackled the problem of quantifying spatial dependence in linguistic variables by developing a new method based on Reproducing Kernel Hilbert space representations, which avoids parametric assumptions and works across diverse data types, showing robust performance on synthetic and real datasets including Dutch tweets and North American newspapers.
Quantifying the degree of spatial dependence for linguistic variables is a key task for analyzing dialectal variation. However, existing approaches have important drawbacks. First, they are based on parametric models of dependence, which limits their power in cases where the underlying parametric assumptions are violated. Second, they are not applicable to all types of linguistic data: some approaches apply only to frequencies, others to boolean indicators of whether a linguistic variable is present. We present a new method for measuring geographical language variation, which solves both of these problems. Our approach builds on Reproducing Kernel Hilbert space (RKHS) representations for nonparametric statistics, and takes the form of a test statistic that is computed from pairs of individual geotagged observations without aggregation into predefined geographical bins. We compare this test with prior work using synthetic data as well as a diverse set of real datasets: a corpus of Dutch tweets, a Dutch syntactic atlas, and a dataset of letters to the editor in North American newspapers. Our proposed test is shown to support robust inferences across a broad range of scenarios and types of data.