Conjunction Subspaces Test for Conformal and Selective Classification
This work addresses uncertainty quantification for classification tasks, offering a method for conformal prediction and selective classification, but it appears incremental as it builds on existing subspace and hypothesis testing frameworks.
The paper tackles the problem of quantifying uncertainty in classification decisions by proposing a new classifier that integrates significance testing over random subspaces to produce consensus p-values, with theoretical generalization error bounds and empirical validation on real datasets showing its effectiveness.
In this paper, we present a new classifier, which integrates significance testing results over different random subspaces to yield consensus p-values for quantifying the uncertainty of classification decision. The null hypothesis is that the test sample has no association with the target class on a randomly chosen subspace, and hence the classification problem can be formulated as a problem of testing for the conjunction of hypotheses. The proposed classifier can be easily deployed for the purpose of conformal prediction and selective classification with reject and refine options by simply thresholding the consensus p-values. The theoretical analysis on the generalization error bound of the proposed classifier is provided and empirical studies on real data sets are conducted as well to demonstrate its effectiveness.