Individual Calibration with Randomized Forecasting
This addresses fairness and vulnerability issues in machine learning by ensuring calibration at the individual level, which is an incremental improvement over existing average calibration methods.
The paper tackles the problem of achieving calibrated predictions for individual samples in regression, rather than on average, by introducing randomized credible intervals. The result is models that are more calibrated for arbitrary subgroups and offer higher utility against adversaries exploiting miscalibration.
Machine learning applications often require calibrated predictions, e.g. a 90\% credible interval should contain the true outcome 90\% of the times. However, typical definitions of calibration only require this to hold on average, and offer no guarantees on predictions made on individual samples. Thus, predictions can be systematically over or under confident on certain subgroups, leading to issues of fairness and potential vulnerabilities. We show that calibration for individual samples is possible in the regression setup if the predictions are randomized, i.e. outputting randomized credible intervals. Randomization removes systematic bias by trading off bias with variance. We design a training objective to enforce individual calibration and use it to train randomized regression functions. The resulting models are more calibrated for arbitrarily chosen subgroups of the data, and can achieve higher utility in decision making against adversaries that exploit miscalibrated predictions.