C-MI-GAN : Estimation of Conditional Mutual Information using MinMax formulation
This work addresses the need for accurate CMI estimation, which is crucial for applications like conditional independence testing, but it is incremental as it builds on existing neural estimators with a new optimization approach.
The paper tackled the problem of estimating conditional mutual information (CMI) by formulating it as a minmax optimization problem, similar to generative adversarial networks, and found that their estimator provided better estimates on simulated data and outperformed state-of-the-art methods in conditional independence testing on real data.
Estimation of information theoretic quantities such as mutual information and its conditional variant has drawn interest in recent times owing to their multifaceted applications. Newly proposed neural estimators for these quantities have overcome severe drawbacks of classical $k$NN-based estimators in high dimensions. In this work, we focus on conditional mutual information (CMI) estimation by utilizing its formulation as a minmax optimization problem. Such a formulation leads to a joint training procedure similar to that of generative adversarial networks. We find that our proposed estimator provides better estimates than the existing approaches on a variety of simulated data sets comprising linear and non-linear relations between variables. As an application of CMI estimation, we deploy our estimator for conditional independence (CI) testing on real data and obtain better results than state-of-the-art CI testers.