Conditional Independence Testing via Latent Representation Learning
This addresses a key problem in causal discovery and statistical analysis for researchers and practitioners, though it appears to be an incremental improvement over existing methods.
The paper tackles conditional independence testing by introducing LCIT, a non-parametric method that learns latent representations of variables to remove information about conditioning variables, then tests for dependencies. The method outperforms state-of-the-art baselines across various metrics and adapts well to non-linear and high-dimensional settings.
Detecting conditional independencies plays a key role in several statistical and machine learning tasks, especially in causal discovery algorithms. In this study, we introduce LCIT (Latent representation based Conditional Independence Test)-a novel non-parametric method for conditional independence testing based on representation learning. Our main contribution involves proposing a generative framework in which to test for the independence between X and Y given Z, we first learn to infer the latent representations of target variables X and Y that contain no information about the conditioning variable Z. The latent variables are then investigated for any significant remaining dependencies, which can be performed using the conventional partial correlation test. The empirical evaluations show that LCIT outperforms several state-of-the-art baselines consistently under different evaluation metrics, and is able to adapt really well to both non-linear and high-dimensional settings on a diverse collection of synthetic and real data sets.