Exact Minimum-Volume Confidence Set Intersection for Multinomial Outcomes
This addresses a practical decision problem in data science for A/B testing and reinforcement learning, offering a certified method to handle the irregular geometry of MVCs, though it is incremental as it builds on existing MVC concepts.
The paper tackles the problem of determining whether minimum-volume confidence sets (MVCs) for multinomial outcomes intersect, which is crucial for A/B testing and reinforcement learning, by presenting a certified algorithm that efficiently certifies intersection, disjointness, or returns an indeterminate result with a prescribed margin, demonstrating reliable decision procedures for core tasks.
Computation of confidence sets is central to data science and machine learning, serving as the workhorse of A/B testing and underpinning the operation and analysis of reinforcement learning algorithms. Among all valid confidence sets for the multinomial parameter, minimum-volume confidence sets (MVCs) are optimal in that they minimize average volume, but they are defined as level sets of an exact p-value that is discontinuous and difficult to compute. Rather than attempting to characterize the geometry of MVCs directly, this paper studies a practically motivated decision problem: given two observed multinomial outcomes, can one certify whether their MVCs intersect? We present a certified, tolerance-aware algorithm for this intersection problem. The method exploits the fact that likelihood ordering induces halfspace constraints in log-odds coordinates, enabling adaptive geometric partitioning of parameter space and computable lower and upper bounds on p-values over each cell. For three categories, this yields an efficient and provably sound algorithm that either certifies intersection, certifies disjointness, or returns an indeterminate result when the decision lies within a prescribed margin. We further show how the approach extends to higher dimensions. The results demonstrate that, despite their irregular geometry, MVCs admit reliable certified decision procedures for core tasks in A/B testing.