Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects
This work addresses the need for robust evaluation metrics in causal inference and personalized medicine, offering a method applicable across randomized and observational studies, though it is incremental as it builds on existing metrics like the Qini coefficient.
The authors tackled the problem of evaluating treatment prioritization rules by proposing rank-weighted average treatment effect (RATE) metrics, which provide a simple and general way to compare and test how well rules identify individuals who benefit most from treatment, with applications including aspirin targeting for stroke patients.
There are a number of available methods for selecting whom to prioritize for treatment, including ones based on treatment effect estimation, risk scoring, and hand-crafted rules. We propose rank-weighted average treatment effect (RATE) metrics as a simple and general family of metrics for comparing and testing the quality of treatment prioritization rules. RATE metrics are agnostic as to how the prioritization rules were derived, and only assess how well they identify individuals that benefit the most from treatment. We define a family of RATE estimators and prove a central limit theorem that enables asymptotically exact inference in a wide variety of randomized and observational study settings. RATE metrics subsume a number of existing metrics, including the Qini coefficient, and our analysis directly yields inference methods for these metrics. We showcase RATE in the context of a number of applications, including optimal targeting of aspirin to stroke patients.