Task-specific experimental design for treatment effect estimation
This addresses the high cost and inefficiency of RCTs for practitioners in fields like marketing, though it is an incremental improvement over existing sample-efficient alternatives.
The paper tackles the problem of expensive and inefficient randomized controlled trials (RCTs) for causal inference by developing a task-specific experimental design method, which outperforms benchmarks by requiring an order-of-magnitude less data to match RCT performance in tasks like targeted marketing.
Understanding causality should be a core requirement of any attempt to build real impact through AI. Due to the inherent unobservability of counterfactuals, large randomised trials (RCTs) are the standard for causal inference. But large experiments are generically expensive, and randomisation carries its own costs, e.g. when suboptimal decisions are trialed. Recent work has proposed more sample-efficient alternatives to RCTs, but these are not adaptable to the downstream application for which the causal effect is sought. In this work, we develop a task-specific approach to experimental design and derive sampling strategies customised to particular downstream applications. Across a range of important tasks, real-world datasets, and sample sizes, our method outperforms other benchmarks, e.g. requiring an order-of-magnitude less data to match RCT performance on targeted marketing tasks.