MCGrad: Multicalibration at Web Scale
This addresses the problem of ensuring fair and calibrated predictions in large-scale machine learning systems for industry practitioners, representing a significant advancement over existing methods.
The paper tackles the challenge of applying multicalibration at scale in industry by introducing MCGrad, a novel algorithm that eliminates the need for manually specifying subgroups, scales effectively, and often improves other performance metrics, with deployments at Meta involving hundreds of production models.
We propose MCGrad, a novel and scalable multicalibration algorithm. Multicalibration - calibration in sub-groups of the data - is an important property for the performance of machine learning-based systems. Existing multicalibration methods have thus far received limited traction in industry. We argue that this is because existing methods (1) require such subgroups to be manually specified, which ML practitioners often struggle with, (2) are not scalable, or (3) may harm other notions of model performance such as log loss and Area Under the Precision-Recall Curve (PRAUC). MCGrad does not require explicit specification of protected groups, is scalable, and often improves other ML evaluation metrics instead of harming them. MCGrad has been in production at Meta, and is now part of hundreds of production models. We present results from these deployments as well as results on public datasets.