Sequential estimation of quantiles with applications to A/B-testing and best-arm identification
This work addresses the need for efficient and reliable quantile estimation in real-time applications like A/B testing and bandit problems, offering significant improvements in sample efficiency.
The paper tackles the problem of estimating quantiles from streaming data with confidence intervals valid uniformly over time, achieving the optimal shrinkage rate of √(t⁻¹ log log t) and providing a new algorithm for best-arm identification in multi-armed bandits that reduces sample requirements by factors of 5 to 50 in simulations.
We propose confidence sequences -- sequences of confidence intervals which are valid uniformly over time -- for quantiles of any distribution over a complete, fully-ordered set, based on a stream of i.i.d. observations. We give methods both for tracking a fixed quantile and for tracking all quantiles simultaneously. Specifically, we provide explicit expressions with small constants for intervals whose widths shrink at the fastest possible $\sqrt{t^{-1} \log\log t}$ rate, along with a non-asymptotic concentration inequality for the empirical distribution function which holds uniformly over time with the same rate. The latter strengthens Smirnov's empirical process law of the iterated logarithm and extends the Dvoretzky-Kiefer-Wolfowitz inequality to hold uniformly over time. We give a new algorithm and sample complexity bound for selecting an arm with an approximately best quantile in a multi-armed bandit framework. In simulations, our method requires fewer samples than existing methods by a factor of five to fifty.