Conformal testing in a binary model situation
This addresses the problem of detecting distributional changes in binary data for researchers in statistical testing, but it is incremental as it focuses on improving existing methods.
The paper tackles the problem of testing the IID assumption using conformal prediction in a binary model with unknown changepoints, finding that existing conformal test martingales work well in simple cases but their efficiency can be greatly improved.
Conformal testing is a way of testing the IID assumption based on conformal prediction. The topic of this note is computational evaluation of the performance of conformal testing in a model situation in which IID binary observations generated from a Bernoulli distribution are followed by IID binary observations generated from another Bernoulli distribution, with the parameters of the distributions and changepoint unknown. Existing conformal test martingales can be used for this task and work well in simple cases, but their efficiency can be improved greatly.