Optimal Empirical Risk Minimization under Temporal Distribution Shifts
This addresses the challenge of deploying models in dynamically evolving environments, offering a theoretically grounded solution that outperforms standard weighting strategies, though it appears incremental as it builds on existing risk minimization methods.
The paper tackles the problem of machine learning under temporal distribution shifts by introducing RIDER, an optimally-weighted empirical risk minimization framework, which consistently improves out-of-sample predictive performance on datasets like Yearbook, stock market volatility, and NYC taxi ride durations.
Temporal distribution shifts pose a key challenge for machine learning models trained and deployed in dynamically evolving environments. This paper introduces RIDER (RIsk minimization under Dynamically Evolving Regimes) which derives optimally-weighted empirical risk minimization procedures under temporal distribution shifts. Our approach is theoretically grounded in the random distribution shift model, where random shifts arise as a superposition of numerous unpredictable changes in the data-generating process. We show that common weighting schemes, such as pooling all data, exponentially weighting data, and using only the most recent data, emerge naturally as special cases in our framework. We demonstrate that RIDER consistently improves out-of-sample predictive performance when applied as a fine-tuning step on the Yearbook dataset, across a range of benchmark methods in Wild-Time. Moreover, we show that RIDER outperforms standard weighting strategies in two other real-world tasks: predicting stock market volatility and forecasting ride durations in NYC taxi data.