Conformal Prediction Sets for Instance Segmentation
This addresses the problem of unreliable predictions in instance segmentation for users in fields like agriculture and medical imaging, offering a method with theoretical guarantees, though it is incremental as it adapts conformal prediction to a specific task.
The paper tackles the lack of principled uncertainty quantification in instance segmentation models by introducing a conformal prediction algorithm that generates adaptive confidence sets with provable guarantees for high Intersection-Over-Union (IoU) with true masks, achieving target coverage and outperforming existing baselines in applications like agricultural field delineation and cell segmentation.
Current instance segmentation models achieve high performance on average predictions, but lack principled uncertainty quantification: their outputs are not calibrated, and there is no guarantee that a predicted mask is close to the ground truth. To address this limitation, we introduce a conformal prediction algorithm to generate adaptive confidence sets for instance segmentation. Given an image and a pixel coordinate query, our algorithm generates a confidence set of instance predictions for that pixel, with a provable guarantee for the probability that at least one of the predictions has high Intersection-Over-Union (IoU) with the true object instance mask. We apply our algorithm to instance segmentation examples in agricultural field delineation, cell segmentation, and vehicle detection. Empirically, we find that our prediction sets vary in size based on query difficulty and attain the target coverage, outperforming existing baselines such as Learn Then Test, Conformal Risk Control, and morphological dilation-based methods. We provide versions of the algorithm with asymptotic and finite sample guarantees.