Predicting Practically? Domain Generalization for Predictive Analytics in Real-world Environments
This work addresses performance degradation in predictive analytics for CRM due to dynamic customer behaviors, offering a method to improve robustness in real-world environments, though it appears incremental as it builds on existing frameworks.
The paper tackles the problem of predictive models degrading due to distribution shifts in customer relationship management by proposing a domain generalization method based on Distributionally Robust Optimization, demonstrating its effectiveness on a real-world customer churn dataset in temporal and spatial settings.
Predictive machine learning models are widely used in customer relationship management (CRM) to forecast customer behaviors and support decision-making. However, the dynamic nature of customer behaviors often results in significant distribution shifts between training data and serving data, leading to performance degradation in predictive models. Domain generalization, which aims to train models that can generalize to unseen environments without prior knowledge of their distributions, has become a critical area of research. In this work, we propose a novel domain generalization method tailored to handle complex distribution shifts, encompassing both covariate and concept shifts. Our method builds upon the Distributionally Robust Optimization framework, optimizing model performance over a set of hypothetical worst-case distributions rather than relying solely on the training data. Through simulation experiments, we demonstrate the working mechanism of the proposed method. We also conduct experiments on a real-world customer churn dataset, and validate its effectiveness in both temporal and spatial generalization settings. Finally, we discuss the broader implications of our method for advancing Information Systems (IS) design research, particularly in building robust predictive models for dynamic managerial environments.