Likelihood-Free Overcomplete ICA and Applications in Causal Discovery
This work addresses the problem of making causal discovery more practical and robust for researchers and practitioners by overcoming computational bottlenecks and parametric constraints in overcomplete ICA, though it is incremental as it builds on existing OICA frameworks.
The paper tackles the limitations of existing overcomplete independent component analysis (OICA) methods, which rely on strong parametric assumptions and computationally expensive EM procedures, by proposing a likelihood-free OICA algorithm (LFOICA) that estimates the mixing matrix directly via back-propagation without density assumptions, resulting in improved computational efficiency and efficacy in causal discovery tasks on synthetic and real data.
Causal discovery witnessed significant progress over the past decades. In particular, many recent causal discovery methods make use of independent, non-Gaussian noise to achieve identifiability of the causal models. Existence of hidden direct common causes, or confounders, generally makes causal discovery more difficult; whenever they are present, the corresponding causal discovery algorithms can be seen as extensions of overcomplete independent component analysis (OICA). However, existing OICA algorithms usually make strong parametric assumptions on the distribution of independent components, which may be violated on real data, leading to sub-optimal or even wrong solutions. In addition, existing OICA algorithms rely on the Expectation Maximization (EM) procedure that requires computationally expensive inference of the posterior distribution of independent components. To tackle these problems, we present a Likelihood-Free Overcomplete ICA algorithm (LFOICA) that estimates the mixing matrix directly by back-propagation without any explicit assumptions on the density function of independent components. Thanks to its computational efficiency, the proposed method makes a number of causal discovery procedures much more practically feasible. For illustrative purposes, we demonstrate the computational efficiency and efficacy of our method in two causal discovery tasks on both synthetic and real data.