Business Policy Experiments using Fractional Factorial Designs: Consumer Retention on DoorDash
This addresses the need for more efficient experimentation in business policy optimization, particularly for large platforms like DoorDash, though it appears incremental as it builds on existing factorial designs and heterogeneous treatment effect estimation methods.
The paper tackles the problem of slow and expensive business decision-making by factorizing business policies and using fractional factorial experimental designs for evaluation, demonstrating a policy that achieves 5% incremental profit at 67% lower implementation cost in consumer promotion at DoorDash.
This paper investigates an approach to both speed up business decision-making and lower the cost of learning through experimentation by factorizing business policies and employing fractional factorial experimental designs for their evaluation. We illustrate how this method integrates with advances in the estimation of heterogeneous treatment effects, elaborating on its advantages and foundational assumptions. We empirically demonstrate the implementation and benefits of our approach and assess its validity in evaluating consumer promotion policies at DoorDash, which is one of the largest delivery platforms in the US. Our approach discovers a policy with 5% incremental profit at 67% lower implementation cost.