A Relative Error-Based Evaluation Framework of Heterogeneous Treatment Effect Estimators
This work addresses the underdeveloped evaluation of HTE estimators for researchers and practitioners in causal inference, but it is incremental as it builds on existing HTE estimation methods.
The authors tackled the problem of evaluating heterogeneous treatment effect (HTE) estimators by proposing a robust evaluation framework based on relative error, which quantifies performance differences between estimators and supports reliable comparisons, and they also introduced a new HTE learning algorithm that leverages this framework and demonstrates desirable performance in experiments.
While significant progress has been made in heterogeneous treatment effect (HTE) estimation, the evaluation of HTE estimators remains underdeveloped. In this article, we propose a robust evaluation framework based on relative error, which quantifies performance differences between two HTE estimators. We first derive the key theoretical conditions on the nuisance parameters that are necessary to achieve a robust estimator of relative error. Building on these conditions, we introduce novel loss functions and design a neural network architecture to estimate nuisance parameters and obtain robust estimation of relative error, thereby achieving reliable evaluation of HTE estimators. We provide the large sample properties of the proposed relative error estimator. Furthermore, beyond evaluation, we propose a new learning algorithm for HTE that leverages both the previously HTE estimators and the nuisance parameters learned through our neural network architecture. Extensive experiments demonstrate that our evaluation framework supports reliable comparisons across HTE estimators, and the proposed learning algorithm for HTE exhibits desirable performance.