A Novel Approach to the Partial Information Decomposition
This work addresses a foundational issue in information theory for researchers, though it appears incremental as it builds on existing concepts like the Blackwell order.
The paper tackles the partial information decomposition problem by proposing a general framework based on set theory analogies and an ordering relation, and applies it using the Blackwell order to overcome drawbacks of previous methods.
We consider the "partial information decomposition" (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals.