Mutual Information Estimation via Normalizing Flows
This addresses the challenge of accurate MI estimation for researchers in machine learning, though it appears incremental as it builds on existing normalizing flow techniques.
The paper tackles the problem of mutual information estimation by introducing estimators based on normalizing flows that map data to distributions where MI is easier to estimate, achieving practical advantages in experiments with high-dimensional data.
We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method.