Discovering Causal Structure with Reproducing-Kernel Hilbert Space $ε$-Machines
This work addresses the challenge of causal inference for researchers in fields like machine learning and complex systems, offering a broadly applicable approach that is incremental in combining existing techniques.
The paper tackles the problem of inferring causal structure from observational data across discrete or continuous events and time by merging computational mechanics with RKHS representation inference, resulting in a method that robustly estimates causal structure and predicts system dynamics even in the presence of noise and high-dimensional data.
We merge computational mechanics' definition of causal states (predictively-equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely-applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time. A structural representation -- a finite- or infinite-state kernel $ε$-machine -- is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker-Plank equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably-infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally-driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high dimensional data.