ProCause: Generating Counterfactual Outcomes to Evaluate Prescriptive Process Monitoring Methods
This work provides a more reliable evaluation framework for Prescriptive Process Monitoring, which is incremental as it builds on existing causal inference methods to better handle temporal dependencies in process data.
The paper tackled the challenge of evaluating Prescriptive Process Monitoring methods by addressing limitations in existing generative approaches, introducing ProCause which integrates multiple causal inference architectures and sequential models, and found that an ensemble approach and LSTMs improve reliability and effectiveness in both simulated and real-world data.
Prescriptive Process Monitoring (PresPM) is the subfield of Process Mining that focuses on optimizing processes through real-time interventions based on event log data. Evaluating PresPM methods is challenging due to the lack of ground-truth outcomes for all intervention actions in datasets. A generative deep learning approach from the field of Causal Inference (CI), RealCause, has been commonly used to estimate the outcomes for proposed intervention actions to evaluate a new policy. However, RealCause overlooks the temporal dependencies in process data, and relies on a single CI model architecture, TARNet, limiting its effectiveness. To address both shortcomings, we introduce ProCause, a generative approach that supports both sequential (e.g., LSTMs) and non-sequential models while integrating multiple CI architectures (S-Learner, T-Learner, TARNet, and an ensemble). Our research using a simulator with known ground truths reveals that TARNet is not always the best choice; instead, an ensemble of models offers more consistent reliability, and leveraging LSTMs shows potential for improved evaluations when temporal dependencies are present. We further validate ProCause's practical effectiveness through a real-world data analysis, ensuring a more reliable evaluation of PresPM methods.