Inductive Conformal Prediction: A Straightforward Introduction with Examples in Python
It addresses the need for reliable uncertainty quantification in critical domains like healthcare, though it is an introductory tutorial rather than a novel research contribution.
The paper introduces Inductive Conformal Prediction (ICP) as a method to provide prediction sets with guaranteed coverage for high-risk applications, such as medical diagnosis, instead of single point predictions.
Inductive Conformal Prediction (ICP) is a set of distribution-free and model agnostic algorithms devised to predict with a user-defined confidence with coverage guarantee. Instead of having point predictions, i.e., a real number in the case of regression or a single class in multi class classification, models calibrated using ICP output an interval or a set of classes, respectively. ICP takes special importance in high-risk settings where we want the true output to belong to the prediction set with high probability. As an example, a classification model might output that given a magnetic resonance image a patient has no latent diseases to report. However, this model output was based on the most likely class, the second most likely class might tell that the patient has a 15% chance of brain tumor or other severe disease and therefore further exams should be conducted. Using ICP is therefore way more informative and we believe that should be the standard way of producing forecasts. This paper is a hands-on introduction, this means that we will provide examples as we introduce the theory.