Towards a pretrained deep learning estimator of the Linfoot informational correlation
This work provides a more accurate estimator for mutual information, which is useful for researchers in statistics and machine learning, though it is incremental as it builds on existing neural estimators with specific copula-based training.
The authors tackled the problem of estimating mutual information between continuous random variables by developing a supervised deep-learning approach using Linfoot informational correlation as labels, showing generally lower bias and variance compared to existing estimators.
We develop a supervised deep-learning approach to estimate mutual information between two continuous random variables. As labels, we use the Linfoot informational correlation, a transformation of mutual information that has many important properties. Our method is based on ground truth labels for Gaussian and Clayton copulas. We compare our method with estimators based on kernel density, k-nearest neighbours and neural estimators. We show generally lower bias and lower variance. As a proof of principle, future research could look into training the model with a more diverse set of examples from other copulas for which ground truth labels are available.