General and Estimable Learning Bound Unifying Covariate and Concept Shifts
This work addresses a core challenge in machine learning by providing a rigorous and general tool for analyzing learning error under distribution shift, which is incremental in improving theoretical bounds and practical estimation methods.
The paper tackled the problem of generalization under distribution shift by showing that existing learning bounds are loose and non-estimable due to support mismatches, and proposed new definitions and a unified error bound using entropic optimal transport, along with estimators and an algorithm to quantify shifts and estimate the bound in practical applications.
Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples. In this paper, we bridge the gap between theory and practical applications. We first show that existing bounds become loose and non-estimable because their concept shift definition breaks when the source and target supports mismatch. Leveraging entropic optimal transport, we propose new support-agnostic definitions for covariate and concept shifts, and derive a novel unified error bound that applies to broad loss functions, label spaces, and stochastic labeling. We further develop estimators for these shifts with concentration guarantees, and the DataShifts algorithm, which can quantify distribution shifts and estimate the error bound in most applications -- a rigorous and general tool for analyzing learning error under distribution shift.