Learning interpretable causal networks from very large datasets, application to 400,000 medical records of breast cancer patients
This work addresses the problem of scalable and reliable causal discovery for researchers in fields like healthcare, enabling insights from large datasets, though it appears incremental as it builds on existing mutual information principles.
The authors tackled the challenge of learning interpretable causal networks from large observational datasets, and developed iMIIC, a method that achieved over 90% correctness in predicted causal effects when applied to nearly 400,000 breast cancer patient records.
Discovering causal effects is at the core of scientific investigation but remains challenging when only observational data is available. In practice, causal networks are difficult to learn and interpret, and limited to relatively small datasets. We report a more reliable and scalable causal discovery method (iMIIC), based on a general mutual information supremum principle, which greatly improves the precision of inferred causal relations while distinguishing genuine causes from putative and latent causal effects. We showcase iMIIC on synthetic and real-life healthcare data from 396,179 breast cancer patients from the US Surveillance, Epidemiology, and End Results program. More than 90\% of predicted causal effects appear correct, while the remaining unexpected direct and indirect causal effects can be interpreted in terms of diagnostic procedures, therapeutic timing, patient preference or socio-economic disparity. iMIIC's unique capabilities open up new avenues to discover reliable and interpretable causal networks across a range of research fields.