Measuring the effects of confounders in medical supervised classification problems: the Confounding Index (CI)
This addresses the issue of misleading performance estimates in biomedical ML due to unknown confounders, offering a tool for quantitative comparison and correction, though it is incremental as it builds on existing strategies.
The paper tackles the problem of confounders in medical supervised classification by introducing the Confounding Index (CI), a novel metric that measures confounding effects without prior knowledge, validated on simulated and real-world neuroimaging data.
Over the years, there has been growing interest in using Machine Learning techniques for biomedical data processing. When tackling these tasks, one needs to bear in mind that biomedical data depends on a variety of characteristics, such as demographic aspects (age, gender, etc) or the acquisition technology, which might be unrelated with the target of the analysis. In supervised tasks, failing to match the ground truth targets with respect to such characteristics, called confounders, may lead to very misleading estimates of the predictive performance. Many strategies have been proposed to handle confounders, ranging from data selection, to normalization techniques, up to the use of training algorithm for learning with imbalanced data. However, all these solutions require the confounders to be known a priori. To this aim, we introduce a novel index that is able to measure the confounding effect of a data attribute in a bias-agnostic way. This index can be used to quantitatively compare the confounding effects of different variables and to inform correction methods such as normalization procedures or ad-hoc-prepared learning algorithms. The effectiveness of this index is validated on both simulated data and real-world neuroimaging data.