Robust Hypothesis Test for Nonlinear Effect with Gaussian Processes
This work addresses the challenge of robust statistical inference for nonlinear effects in machine learning, though it appears incremental as it builds on existing kernel and hypothesis testing frameworks.
The authors tackled the problem of detecting whether a data-generating function belongs to a partially known reproducing kernel Hilbert space by developing a robust hypothesis test using Gaussian processes, which they applied to detect nonlinear interactions between features and evaluated its finite-sample performance.
This work constructs a hypothesis test for detecting whether an data-generating function $h: R^p \rightarrow R$ belongs to a specific reproducing kernel Hilbert space $\mathcal{H}_0$ , where the structure of $\mathcal{H}_0$ is only partially known. Utilizing the theory of reproducing kernels, we reduce this hypothesis to a simple one-sided score test for a scalar parameter, develop a testing procedure that is robust against the mis-specification of kernel functions, and also propose an ensemble-based estimator for the null model to guarantee test performance in small samples. To demonstrate the utility of the proposed method, we apply our test to the problem of detecting nonlinear interaction between groups of continuous features. We evaluate the finite-sample performance of our test under different data-generating functions and estimation strategies for the null model. Our results reveal interesting connections between notions in machine learning (model underfit/overfit) and those in statistical inference (i.e. Type I error/power of hypothesis test), and also highlight unexpected consequences of common model estimating strategies (e.g. estimating kernel hyperparameters using maximum likelihood estimation) on model inference.