FMMI: Flow Matching Mutual Information Estimation
This addresses the challenge of accurate and scalable mutual information estimation for machine learning and data analysis, representing a significant advancement rather than an incremental improvement.
The paper tackled the problem of mutual information estimation by introducing a novel estimator that reframes the discriminative approach, learning a normalizing flow to transform distributions, resulting in a computationally efficient and precise method that scales well to high dimensions and across a wide range of MI values.
We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other. This technique produces a computationally efficient and precise MI estimate that scales well to high dimensions and across a wide range of ground-truth MI values.