Finite-State Extreme Effect Variable
This work addresses the need for more precise causal inference in controlled experiments like A/B testing, though it appears incremental as it generalizes an existing notion to finite states.
The authors tackled the problem of identifying an extreme effect variable that accumulates all the effect of a variant variable on another observable variable in a finite-state setting, showing its utility in online A/B testing for web search engine evaluation.
We generalize to the finite-state case the notion of the extreme effect variable $Y$ that accumulates all the effect of a variant variable $V$ observed in changes of another variable $X$. We conduct theoretical analysis and turn the problem of finding of an effect variable into a problem of a simultaneous decomposition of a set of distributions. The states of the extreme effect variable, on the one hand, are minimally affected by the variant variable $V$ and, on the other hand, are extremely different with respect to the observable variable $X$. We apply our technique to online evaluation of a web search engine through A/B testing and show its utility.