Kernel Tests of Equivalence
This addresses the need for non-parametric equivalence testing for full distributions, which is important for statisticians and researchers in fields like machine learning, though it is incremental as it builds on existing kernel methods.
The authors tackled the problem of assessing equivalence between distributions, where traditional tests fail to confirm absence of differences due to Type-II errors, by proposing kernel-based tests using kernel Stein discrepancy and Maximum Mean Discrepancy, with results validated through numerical experiments.
We propose novel kernel-based tests for assessing the equivalence between distributions. Traditional goodness-of-fit testing is inappropriate for concluding the absence of distributional differences, because failure to reject the null hypothesis may simply be a result of lack of test power, also known as the Type-II error. This motivates \emph{equivalence testing}, which aims to assess the \emph{absence} of a statistically meaningful effect under controlled error rates. However, existing equivalence tests are either limited to parametric distributions or focus only on specific moments rather than the full distribution. We address these limitations using two kernel-based statistical discrepancies: the \emph{kernel Stein discrepancy} and the \emph{Maximum Mean Discrepancy}. The null hypothesis of our proposed tests assumes the candidate distribution differs from the nominal distribution by at least a pre-defined margin, which is measured by these discrepancies. We propose two approaches for computing the critical values of the tests, one using an asymptotic normality approximation, and another based on bootstrapping. Numerical experiments are conducted to assess the performance of these tests.