It's Hard to Be Normal: The Impact of Noise on Structure-agnostic Estimation
This work addresses a key limitation in causal inference for researchers and practitioners, offering incremental improvements by developing more robust estimators for non-Gaussian noise scenarios.
The paper investigates how treatment noise distribution affects the performance of structure-agnostic causal inference, showing that the double machine learning estimator is optimal for Gaussian noise but suboptimal for non-Gaussian noise, and introduces new procedures with higher-order robustness that achieve practical gains in synthetic experiments.
Structure-agnostic causal inference studies how well one can estimate a treatment effect given black-box machine learning estimates of nuisance functions (like the impact of confounders on treatment and outcomes). Here, we find that the answer depends in a surprising way on the distribution of the treatment noise. Focusing on the partially linear model of \citet{robinson1988root}, we first show that the widely adopted double machine learning (DML) estimator is minimax rate-optimal for Gaussian treatment noise, resolving an open problem of \citet{mackey2018orthogonal}. Meanwhile, for independent non-Gaussian treatment noise, we show that DML is always suboptimal by constructing new practical procedures with higher-order robustness to nuisance errors. These \emph{ACE} procedures use structure-agnostic cumulant estimators to achieve $r$-th order insensitivity to nuisance errors whenever the $(r+1)$-st treatment cumulant is non-zero. We complement these core results with novel minimax guarantees for binary treatments in the partially linear model. Finally, using synthetic demand estimation experiments, we demonstrate the practical benefits of our higher-order robust estimators.