Anytime-valid, Bayes-assisted, Prediction-Powered Inference
This work addresses the need for efficient and valid statistical inference in dynamic data environments, such as online learning or streaming applications, offering a method that is incremental by building on existing prediction-powered inference frameworks.
The paper tackles the problem of constructing confidence intervals in sequential settings where labeled and unlabeled data accumulate over time, by extending prediction-powered inference to achieve anytime-valid confidence sequences that improve statistical efficiency while maintaining validity uniformly over time.
Given a large pool of unlabelled data and a smaller amount of labels, prediction-powered inference (PPI) leverages machine learning predictions to increase the statistical efficiency of confidence interval procedures based solely on labelled data, while preserving fixed-time validity. In this paper, we extend the PPI framework to the sequential setting, where labelled and unlabelled datasets grow over time. Exploiting Ville's inequality and the method of mixtures, we propose prediction-powered confidence sequence procedures that are asymptotically valid uniformly over time and naturally accommodate prior knowledge on the quality of the predictions to further boost efficiency. We carefully illustrate the design choices behind our method and demonstrate its effectiveness in real and synthetic examples.