Beyond the Best: Estimating Distribution Functionals in Infinite-Armed Bandits
This work addresses a general problem in bandit theory for researchers, extending beyond best-arm identification to broader distributional analysis, though it is incremental in building on prior work on maximum estimation.
The paper tackles the problem of estimating distribution functionals beyond the maximum in infinite-armed bandits, proposing unified meta algorithms for offline and online settings that achieve optimal sample complexities, with online estimation reducing sample complexity for functionals like the median, maximum, and trimmed mean.
In the infinite-armed bandit problem, each arm's average reward is sampled from an unknown distribution, and each arm can be sampled further to obtain noisy estimates of the average reward of that arm. Prior work focuses on identifying the best arm, i.e., estimating the maximum of the average reward distribution. We consider a general class of distribution functionals beyond the maximum, and propose unified meta algorithms for both the offline and online settings, achieving optimal sample complexities. We show that online estimation, where the learner can sequentially choose whether to sample a new or existing arm, offers no advantage over the offline setting for estimating the mean functional, but significantly reduces the sample complexity for other functionals such as the median, maximum, and trimmed mean. The matching lower bounds utilize several different Wasserstein distances. For the special case of median estimation, we identify a curious thresholding phenomenon on the indistinguishability between Gaussian convolutions with respect to the noise level, which may be of independent interest.