Information Theoretic Structure Learning with Confidence
This work addresses a bottleneck in statistical learning for researchers analyzing complex, continuous data, offering a more efficient and statistically rigorous approach to structure discovery.
The paper tackles the problem of slow convergence in nonparametric structure discovery for continuous multivariate models by introducing a new method using weighted ensemble divergence estimators, achieving parametric convergence rates and enabling statistical validation through an asymptotic central limit theorem.
Information theoretic measures (e.g. the Kullback Liebler divergence and Shannon mutual information) have been used for exploring possibly nonlinear multivariate dependencies in high dimension. If these dependencies are assumed to follow a Markov factor graph model, this exploration process is called structure discovery. For discrete-valued samples, estimates of the information divergence over the parametric class of multinomial models lead to structure discovery methods whose mean squared error achieves parametric convergence rates as the sample size grows. However, a naive application of this method to continuous nonparametric multivariate models converges much more slowly. In this paper we introduce a new method for nonparametric structure discovery that uses weighted ensemble divergence estimators that achieve parametric convergence rates and obey an asymptotic central limit theorem that facilitates hypothesis testing and other types of statistical validation.