Research Reproducibility as a Survival Analysis
This work addresses the problem of accurately modeling research reproducibility for the machine learning community, offering a more nuanced understanding than current binary approaches.
The paper proposes modeling research reproducibility as a survival analysis problem, moving away from a binary classification. This new perspective allows for drawing novel insights that better explain existing longitudinal data on reproducibility.
There has been increasing concern within the machine learning community that we are in a reproducibility crisis. As many have begun to work on this problem, all work we are aware of treat the issue of reproducibility as an intrinsic binary property: a paper is or is not reproducible. Instead, we consider modeling the reproducibility of a paper as a survival analysis problem. We argue that this perspective represents a more accurate model of the underlying meta-science question of reproducible research, and we show how a survival analysis allows us to draw new insights that better explain prior longitudinal data. The data and code can be found at https://github.com/EdwardRaff/Research-Reproducibility-Survival-Analysis