S$Ω$I: Score-based O-INFORMATION Estimation
This enables more practical analysis of complex systems in fields like neuroscience or climate science, though it is incremental as it builds on existing O-information theory.
The paper tackles the problem of estimating O-information, a measure for high-order dependencies in multivariate systems, without restrictive assumptions, and demonstrates its effectiveness on synthetic and real-world data.
The analysis of scientific data and complex multivariate systems requires information quantities that capture relationships among multiple random variables. Recently, new information-theoretic measures have been developed to overcome the shortcomings of classical ones, such as mutual information, that are restricted to considering pairwise interactions. Among them, the concept of information synergy and redundancy is crucial for understanding the high-order dependencies between variables. One of the most prominent and versatile measures based on this concept is O-information, which provides a clear and scalable way to quantify the synergy-redundancy balance in multivariate systems. However, its practical application is limited to simplified cases. In this work, we introduce S$Ω$I, which allows for the first time to compute O-information without restrictive assumptions about the system. Our experiments validate our approach on synthetic data, and demonstrate the effectiveness of S$Ω$I in the context of a real-world use case.