Exploring and measuring non-linear correlations: Copulas, Lightspeed Transportation and Clustering
This work addresses the challenge of analyzing complex dependencies in data for researchers and practitioners in statistics and machine learning, though it appears incremental as it builds on existing techniques like copulas and optimal transport.
The authors tackled the problem of exploring and measuring non-linear correlations between variables by proposing a methodology that combines copulas, optimal transport, and clustering, resulting in a novel dependence coefficient that can target specific patterns, as illustrated and benchmarked on several datasets.
We propose a methodology to explore and measure the pairwise correlations that exist between variables in a dataset. The methodology leverages copulas for encoding dependence between two variables, state-of-the-art optimal transport for providing a relevant geometry to the copulas, and clustering for summarizing the main dependence patterns found between the variables. Some of the clusters centers can be used to parameterize a novel dependence coefficient which can target or forget specific dependence patterns. Finally, we illustrate and benchmark the methodology on several datasets. Code and numerical experiments are available online for reproducible research.