Conserved active information
Provides a new theoretical tool for information theory that resolves a critique of active information, benefiting researchers in search, optimization, and related fields.
The paper introduces conserved active information, a symmetric measure that quantifies net information gain/loss across the entire search space while respecting No-Free-Lunch conservation. It reveals hidden regimes not captured by KL divergence, such as when strong knowledge reduces global disorder, with applications in search, optimization, and cosmology.
We introduce conserved active information $I^\oplus$, a symmetric extension of active information that quantifies net information gain/loss across the entire search space, respecting No-Free-Lunch conservation. Through Bernoulli and uniform-baseline examples, we show $I^\oplus$ reveals regimes hidden from KL divergence, such as when strong knowledge reduces global disorder. Such regimes are proven formally under uniform baseline, distinguishing disorder (increasing mild knowledge from order-imposing strong knowledge. We further illustrate these regimes with examples from Markov chains and cosmological fine-tuning. This resolves a longstanding critique of active information while enabling applications in search, optimization, and beyond.