Explain Variance of Prediction in Variational Time Series Models for Clinical Deterioration Prediction
This work addresses the challenge of balancing measurement costs and predictive accuracy in clinical settings, offering a method to quantify the impact of measurements on uncertainty, though it is incremental as it applies existing SHAP techniques to a new context.
The study tackled the problem of determining optimal clinical variable measurement frequency by linking it to prediction uncertainty reduction in variational time series models, proposing variance SHAP to attribute epistemic uncertainty and testing it on an ICU dataset for deterioration prediction.
Missingness and measurement frequency are two sides of the same coin. How frequent should we measure clinical variables and conduct laboratory tests? It depends on many factors such as the stability of patient conditions, diagnostic process, treatment plan and measurement costs. The utility of measurements varies disease by disease, patient by patient. In this study we propose a novel view of clinical variable measurement frequency from a predictive modeling perspective, namely the measurements of clinical variables reduce uncertainty in model predictions. To achieve this goal, we propose variance SHAP with variational time series models, an application of Shapley Additive Expanation(SHAP) algorithm to attribute epistemic prediction uncertainty. The prediction variance is estimated by sampling the conditional hidden space in variational models and can be approximated deterministically by delta's method. This approach works with variational time series models such as variational recurrent neural networks and variational transformers. Since SHAP values are additive, the variance SHAP of binary data imputation masks can be directly interpreted as the contribution to prediction variance by measurements. We tested our ideas on a public ICU dataset with deterioration prediction task and study the relation between variance SHAP and measurement time intervals.