MAPIE: an open-source library for distribution-free uncertainty quantification
This library addresses the need for robust uncertainty quantification in ML models, making it accessible to practitioners through a scikit-learn-compatible tool, though it is incremental as it builds on existing conformal prediction methods.
The authors tackled the problem of estimating uncertainties in machine learning predictions by introducing MAPIE, an open-source Python library that implements conformal prediction methods for regression and classification, providing strong theoretical guarantees on marginal coverages with mild assumptions.
Estimating uncertainties associated with the predictions of Machine Learning (ML) models is of crucial importance to assess their robustness and predictive power. In this submission, we introduce MAPIE (Model Agnostic Prediction Interval Estimator), an open-source Python library that quantifies the uncertainties of ML models for single-output regression and multi-class classification tasks. MAPIE implements conformal prediction methods, allowing the user to easily compute uncertainties with strong theoretical guarantees on the marginal coverages and with mild assumptions on the model or on the underlying data distribution. MAPIE is hosted on scikit-learn-contrib and is fully "scikit-learn-compatible". As such, it accepts any type of regressor or classifier coming with a scikit-learn API. The library is available at: https://github.com/scikit-learn-contrib/MAPIE/.