Distribution-Free Matrix Prediction Under Arbitrary Missing Pattern
This addresses the open problem of distribution-free matrix prediction for applications requiring reliable uncertainty estimates in incomplete data settings, representing a novel contribution to a little-studied topic.
The paper tackles the problem of conformalized entry prediction in row/column-exchangeable matrices under arbitrary missing patterns, proposing two computationally efficient algorithms that safeguard coverage validity and quantify the impact of missingness on accuracy, with empirical evidence showing superior performance.
This paper studies the open problem of conformalized entry prediction in a row/column-exchangeable matrix. The matrix setting presents novel and unique challenges, but there exists little work on this interesting topic. We meticulously define the problem, differentiate it from closely related problems, and rigorously delineate the boundary between achievable and impossible goals. We then propose two practical algorithms. The first method provides a fast emulation of the full conformal prediction, while the second method leverages the technique of algorithmic stability for acceleration. Both methods are computationally efficient and can effectively safeguard coverage validity in presence of arbitrary missing pattern. Further, we quantify the impact of missingness on prediction accuracy and establish fundamental limit results. Empirical evidence from synthetic and real-world data sets corroborates the superior performance of our proposed methods.