Robust Conformal Prediction under Distribution Shift via Physics-Informed Structural Causal Model
This work addresses reliability in uncertainty quantification for machine learning applications like traffic and epidemiology, offering an incremental improvement over existing conformal prediction methods.
The paper tackles the problem of maintaining coverage guarantees in conformal prediction under conditional distribution shifts by proposing a physics-informed structural causal model (PI-SCM) to reduce coverage loss, validated with real-world datasets showing improved robustness in traffic speed and epidemic spread prediction tasks.
Uncertainty is critical to reliable decision-making with machine learning. Conformal prediction (CP) handles uncertainty by predicting a set on a test input, hoping the set to cover the true label with at least $(1-α)$ confidence. This coverage can be guaranteed on test data even if the marginal distributions $P_X$ differ between calibration and test datasets. However, as it is common in practice, when the conditional distribution $P_{Y|X}$ is different on calibration and test data, the coverage is not guaranteed and it is essential to measure and minimize the coverage loss under distributional shift at \textit{all} possible confidence levels. To address these issues, we upper bound the coverage difference at all levels using the cumulative density functions of calibration and test conformal scores and Wasserstein distance. Inspired by the invariance of physics across data distributions, we propose a physics-informed structural causal model (PI-SCM) to reduce the upper bound. We validated that PI-SCM can improve coverage robustness along confidence level and test domain on a traffic speed prediction task and an epidemic spread task with multiple real-world datasets.