Fast Non-Parametric Tests of Relative Dependency and Similarity
This work addresses the need for robust statistical comparisons in fields like bioinformatics and machine learning, though it is incremental as it builds on existing HSIC and MMD methods.
The paper tackles the problem of comparing dependencies and similarities between variables or distributions by introducing two non-parametric statistical tests: one for relative dependency using HSIC and another for relative similarity using MMD, with results demonstrated on real-world applications such as identifying language groups and analyzing tumor data.
We introduce two novel non-parametric statistical hypothesis tests. The first test, called the relative test of dependency, enables us to determine whether one source variable is significantly more dependent on a first target variable or a second. Dependence is measured via the Hilbert-Schmidt Independence Criterion (HSIC). The second test, called the relative test of similarity, is use to determine which of the two samples from arbitrary distributions is significantly closer to a reference sample of interest and the relative measure of similarity is based on the Maximum Mean Discrepancy (MMD). To construct these tests, we have used as our test statistics the difference of HSIC statistics and of MMD statistics, respectively. The resulting tests are consistent and unbiased, and have favorable convergence properties. The effectiveness of the relative dependency test is demonstrated on several real-world problems: we identify languages groups from a multilingual parallel corpus, and we show that tumor location is more dependent on gene expression than chromosome imbalance. We also demonstrate the performance of the relative test of similarity over a broad selection of model comparisons problems in deep generative models.