Algorithm as Experiment: Machine Learning, Market Design, and Policy Eligibility Rules
This work addresses the challenge of causal inference in policy evaluation for researchers and policymakers, though it is incremental in extending existing methods to algorithmic contexts.
The authors tackled the problem of evaluating causal effects when algorithms determine policy eligibility, developing a consistent estimator for such scenarios and applying it to assess the CARES Act's hospital funding, finding it had little effect on COVID-19 activities.
Algorithms make a growing portion of policy and business decisions. We develop a treatment-effect estimator using algorithmic decisions as instruments for a class of stochastic and deterministic algorithms. Our estimator is consistent and asymptotically normal for well-defined causal effects. A special case of our setup is multidimensional regression discontinuity designs with complex boundaries. We apply our estimator to evaluate the Coronavirus Aid, Relief, and Economic Security Act, which allocated many billions of dollars worth of relief funding to hospitals via an algorithmic rule. The funding is shown to have little effect on COVID-19-related hospital activities. Naive estimates exhibit selection bias.