An Exploration of Multicalibration Uniform Convergence Bounds
This work offers incremental insights for machine learning practitioners in algorithmic fairness by enabling easier understanding of multicalibration error convergence across various classifier architectures.
The paper tackles the problem of understanding the sample complexity for multicalibration error in fair machine learning by developing a framework that reparametrizes ERM sample complexities, showing that this error depends on classifier architecture and data distribution, with experimental evaluation providing insights into convergence behavior.
Recent works have investigated the sample complexity necessary for fair machine learning. The most advanced of such sample complexity bounds are developed by analyzing multicalibration uniform convergence for a given predictor class. We present a framework which yields multicalibration error uniform convergence bounds by reparametrizing sample complexities for Empirical Risk Minimization (ERM) learning. From this framework, we demonstrate that multicalibration error exhibits dependence on the classifier architecture as well as the underlying data distribution. We perform an experimental evaluation to investigate the behavior of multicalibration error for different families of classifiers. We compare the results of this evaluation to multicalibration error concentration bounds. Our investigation provides additional perspective on both algorithmic fairness and multicalibration error convergence bounds. Given the prevalence of ERM sample complexity bounds, our proposed framework enables machine learning practitioners to easily understand the convergence behavior of multicalibration error for a myriad of classifier architectures.