Regression with Cost-based Rejection
This work addresses a novel regression problem for applications requiring reliable predictions, such as safety-critical systems, by extending cost-based rejection from classification to regression, which is incremental but fills a gap in handling continuous target spaces.
The paper tackles the problem of regression with cost-based rejection, where a model can abstain from making predictions on uncertain examples given a rejection cost, and demonstrates that the optimal solution rejects predictions when the variance exceeds the cost, with experiments showing effectiveness.
Learning with rejection is an important framework that can refrain from making predictions to avoid critical mispredictions by balancing between prediction and rejection. Previous studies on cost-based rejection only focused on the classification setting, which cannot handle the continuous and infinite target space in the regression setting. In this paper, we investigate a novel regression problem called regression with cost-based rejection, where the model can reject to make predictions on some examples given certain rejection costs. To solve this problem, we first formulate the expected risk for this problem and then derive the Bayes optimal solution, which shows that the optimal model should reject to make predictions on the examples whose variance is larger than the rejection cost when the mean squared error is used as the evaluation metric. Furthermore, we propose to train the model by a surrogate loss function that considers rejection as binary classification and we provide conditions for the model consistency, which implies that the Bayes optimal solution can be recovered by our proposed surrogate loss. Extensive experiments demonstrate the effectiveness of our proposed method.