Unit Selection: Case Study and Comparison with A/B Test Heuristic
This work addresses decision-making challenges in causal inference for researchers and practitioners, but it is incremental as it builds on existing models.
The paper tackles the unit selection problem for identifying individuals with desired counterfactual behaviors, showing that A/B test heuristics are generally problematic and are special cases of the Li-Pearl benefit function, with simulated use cases provided to aid correct application.
The unit selection problem defined by Li and Pearl identifies individuals who have desired counterfactual behavior patterns, for example, individuals who would respond positively if encouraged and would not otherwise. Li and Pearl showed by example that their unit selection model is beyond the A/B test heuristics. In this paper, we reveal the essence of the A/B test heuristics, which are exceptional cases of the benefit function defined by Li and Pearl. Furthermore, We provided more simulated use cases of Li-Pearl's unit selection model to help decision-makers apply their model correctly, explaining that A/B test heuristics are generally problematic.