Improving reproducibility by controlling random seed stability in machine learning based estimation via bagging
For practitioners using debiased machine learning, this work provides a practical method to improve reproducibility without large computational overhead.
The paper addresses instability in debiased machine learning estimators caused by random seed variation, and proves that subbagging ensures stability for bounded-outcome regression. The proposed adaptive cross-bagging method eliminates seed dependence from both nuisance estimation and sample splitting, achieving targeted stability with small computational cost.
Predictions from machine learning algorithms can vary across random seeds, inducing instability in downstream debiased machine learning estimators. We formalize random seed stability via a concentration condition and prove that subbagging guarantees stability for any bounded-outcome regression algorithm. We introduce a new cross-fitting procedure, adaptive cross-bagging, which simultaneously eliminates seed dependence from both nuisance estimation and sample splitting in debiased machine learning. Numerical experiments confirm that the method achieves the targeted level of stability whereas alternatives do not. Our method incurs a small computational penalty relative to standard practice whereas alternative methods incur large penalties.