ML-PWS: Estimating the Mutual Information Between Experimental Time Series Using Neural Networks
This work addresses the problem of quantifying information transmission in systems like neuroscience or engineering, where experimental data lacks models, offering a practical tool but is incremental as it builds on existing PWS techniques.
The paper tackles the challenge of estimating mutual information from experimental time-series data by introducing ML-PWS, a method that combines machine learning with Path Weight Sampling to compute information rates without requiring a pre-existing mathematical model, achieving accurate results validated on synthetic data and applied to neuronal data.
The ability to quantify information transmission is crucial for the analysis and design of natural and engineered systems. The information transmission rate is the fundamental measure for systems with time-varying signals, yet computing it is extremely challenging. In particular, the rate cannot be obtained directly from experimental time-series data without approximations, because of the high dimensionality of the signal trajectory space. Path Weight Sampling (PWS) is a computational technique that makes it possible to obtain the information rate exactly for any stochastic system. However, it requires a mathematical model of the system of interest, be it described by a master equation or a set of differential equations. Here, we present a technique that employs Machine Learning (ML) to develop a generative model from experimental time-series data, which is then combined with PWS to obtain the information rate. We demonstrate the accuracy of this technique, called ML-PWS, by comparing its results on synthetic time-series data generated from a non-linear model against ground-truth results obtained by applying PWS directly to the same model. We illustrate the utility of ML-PWS by applying it to neuronal time-series data.