Conformal Prediction and Trustworthy AI
It addresses the need for reliable uncertainty quantification in AI to enhance trustworthiness, but is incremental as it builds on existing conformal prediction methods.
The paper reviews how conformal prediction, a method for set predictions with guaranteed confidence levels, can contribute to trustworthy AI by addressing issues like generalization risk and AI governance, with experiments demonstrating its use for calibration and bias mitigation.
Conformal predictors are machine learning algorithms developed in the 1990's by Gammerman, Vovk, and their research team, to provide set predictions with guaranteed confidence level. Over recent years, they have grown in popularity and have become a mainstream methodology for uncertainty quantification in the machine learning community. From its beginning, there was an understanding that they enable reliable machine learning with well-calibrated uncertainty quantification. This makes them extremely beneficial for developing trustworthy AI, a topic that has also risen in interest over the past few years, in both the AI community and society more widely. In this article, we review the potential for conformal prediction to contribute to trustworthy AI beyond its marginal validity property, addressing problems such as generalization risk and AI governance. Experiments and examples are also provided to demonstrate its use as a well-calibrated predictor and for bias identification and mitigation.