Estimating the Value of Evidence-Based Decision Making
This addresses the challenge for organizations in business and policy to empirically evaluate and optimize their evidence-gathering strategies, though it appears incremental as it builds on existing empirical Bayes methods.
The paper tackles the problem of organizations lacking tools to assess the value of evidence-based decision making (EBDM) and optimize statistical precision, introducing an empirical framework that shows commonly used decision rules based on statistical significance can leave substantial value unrealized or generate negative expected value.
In an era of data abundance, statistical evidence is increasingly critical for business and policy decisions. Yet, organizations lack empirical tools to assess the value of evidence-based decision making (EBDM), optimize statistical precision, and balance the costs of evidence-gathering strategies against their benefits. To tackle these challenges, this article introduces an empirical framework to estimate the value of EBDM and evaluate the return on investment in statistical precision and project ideation. The framework leverages parametric and nonparametric empirical Bayes methods to account for parameter heterogeneity and measure how statistical precision changes the value of evidence. The value extracted from statistical evidence depends critically on how organizations translate evidence into policy decisions. Commonly used decision rules based on statistical significance can leave substantial value unrealized and, in some cases, generate negative expected value.