Transfer Entropy: where Shannon meets Turing
This work addresses the challenge of determining causal structures from noisy data for researchers in information theory and time series analysis, but it appears incremental as it builds on existing transfer entropy concepts.
The paper tackled the problem of capturing nonlinear source-destination relations in multi-variate time series using transfer entropy, showing that bivariate analysis suffices to distinguish true from false relations in specific cases like three stochastic processes, and derived the Data Processing Inequality for transfer entropy.
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear transformations between their probability mass functions. The resulting tensor formalism is used to show that in specific cases, e.g., in the case the system consists of three stochastic processes, bivariate analysis suffices to distinguish true relations from false relations. This allows us to determine the causal structure as far as encoded in the probability mass functions of noisy data. The tensor formalism was also used to derive the Data Processing Inequality for transfer entropy.