Entropy Regularization for Population Estimation
This addresses population estimation tasks crucial for public policy under legal constraints, though it appears incremental as it applies a known regularization technique to a specific setting.
The paper tackles the problem of mean reward estimation in structured bandit settings, showing that entropy regularization yields nearly unbiased estimates with lower variance than existing baselines.
Entropy regularization is known to improve exploration in sequential decision-making problems. We show that this same mechanism can also lead to nearly unbiased and lower-variance estimates of the mean reward in the optimize-and-estimate structured bandit setting. Mean reward estimation (i.e., population estimation) tasks have recently been shown to be essential for public policy settings where legal constraints often require precise estimates of population metrics. We show that leveraging entropy and KL divergence can yield a better trade-off between reward and estimator variance than existing baselines, all while remaining nearly unbiased. These properties of entropy regularization illustrate an exciting potential for bridging the optimal exploration and estimation literatures.