Separating Shortcut Transition from Cross-Family OOD Failure in a Minimal Model
For the OOD generalization community, this work clarifies that training correlation, learned shortcut use, and test failure are distinct phenomena, providing a theoretical framework to disentangle them.
The paper analyzes a minimal binary model to separate shortcut attraction, shortcut-rule transition, and cross-family OOD failure, showing that ridge-logistic ERM switches to the shortcut rule when training shortcut signal exceeds invariant signal, with failure depending on the held-out family's correlation sign and strength.
Shortcut features are often invoked to explain out-of-distribution (OOD) failure, but training correlation, learned shortcut use, and test-time failure need not coincide. We study a minimal binary model with one invariant coordinate and one family-dependent shortcut coordinate. In the deterministic regime, positive average shortcut correlation pulls logistic ERM toward positive shortcut weight, but ridge regularization keeps the classifier invariant-dominated and prevents deterministic OOD failure. When the invariant coordinate is noisy, ridge-logistic ERM switches to the shortcut rule once the training shortcut signal exceeds the invariant signal. Whether that transition causes failure depends on the held-out family: weaker shortcut correlation yields positive excess risk, and sign-flipped families yield above-chance error. Synthetic checks match these analytic regimes and show that the same training-side transition can have different held-out consequences. The model separates shortcut attraction, shortcut-rule transition, and cross-family OOD failure.