Neuropathic Pain Diagnosis Simulator for Causal Discovery Algorithm Evaluation
This provides a tool for systematically evaluating causal discovery algorithms in a medical domain, addressing a known bottleneck in the field, though it is incremental as it applies existing simulation concepts to a specific setting.
The authors tackled the problem of evaluating causal discovery algorithms, which lack benchmark datasets with known ground-truth causal relations, by building a neuropathic pain diagnosis simulator that generates data with similar statistics to real-world data and allows infinite samples without privacy concerns.
Discovery of causal relations from observational data is essential for many disciplines of science and real-world applications. However, unlike other machine learning algorithms, whose development has been greatly fostered by a large amount of available benchmark datasets, causal discovery algorithms are notoriously difficult to be systematically evaluated because few datasets with known ground-truth causal relations are available. In this work, we handle the problem of evaluating causal discovery algorithms by building a flexible simulator in the medical setting. We develop a neuropathic pain diagnosis simulator, inspired by the fact that the biological processes of neuropathic pathophysiology are well studied with well-understood causal influences. Our simulator exploits the causal graph of the neuropathic pain pathology and its parameters in the generator are estimated from real-life patient cases. We show that the data generated from our simulator have similar statistics as real-world data. As a clear advantage, the simulator can produce infinite samples without jeopardizing the privacy of real-world patients. Our simulator provides a natural tool for evaluating various types of causal discovery algorithms, including those to deal with practical issues in causal discovery, such as unknown confounders, selection bias, and missing data. Using our simulator, we have evaluated extensively causal discovery algorithms under various settings.