Program Evaluation with Remotely Sensed Outcomes
This addresses a methodological problem for economists and researchers using remote sensing in experiments, offering a correction to a common but flawed practice, though it is incremental in improving existing methods.
The paper tackles bias in program evaluation when using remotely sensed variables (RSVs) like satellite images as proxies for economic outcomes, showing that common practice is biased if RSVs are post-outcome variables, and provides a nonparametric identification method that corrects this without requiring statistical properties of predictions, reanalyzing an anti-poverty program in India.
Economists often estimate treatment effects in experiments using remotely sensed variables (RSVs), e.g., satellite images or mobile phone activity, in place of directly measured economic outcomes. A common practice is to use an observational sample to train a predictor of the economic outcome from the RSV, and then use these predictions as the outcomes in the experiment. We show that this method is biased whenever the RSV is a post-outcome variable, meaning that variation in the economic outcome causes variation in the RSV. For example, changes in poverty or environmental quality cause changes in satellite images, but not vice versa. As our main result, we nonparametrically identify the treatment effect by formalizing the intuition underlying common practice: the conditional distribution of the RSV given the outcome and treatment is stable across samples. Our identifying formula reveals that efficient inference requires predictions of three quantities from the RSV -- the outcome, treatment, and sample indicator -- whereas common practice only predicts the outcome. Valid inference does not require any rate conditions on RSV predictions, justifying the use of complex deep learning algorithms with unknown statistical properties. We reanalyze the effect of an anti-poverty program in India using satellite images.