Proactive Routing to Interpretable Surrogates with Distribution-Free Safety Guarantees
This work addresses the need for safe and interpretable model deployment in machine learning, offering a distribution-free guarantee for practitioners, though it is incremental as it builds on existing conformal methods.
The paper tackles the problem of model routing by proposing a proactive gate-based method with conformal calibration to control the fraction of inputs where a simpler surrogate model's performance degrades beyond a tolerance, achieving controlled violation rates and higher coverage across 35 datasets compared to baselines.
Model routing determines whether to use an accurate black-box model or a simpler surrogate that approximates it at lower cost or greater interpretability. In deployment settings, practitioners often wish to restrict surrogate use to inputs where its degradation relative to a reference model is controlled. We study proactive (input-based) routing, in which a lightweight gate selects the model before either runs, enabling distribution-free control of the fraction of routed inputs whose degradation exceeds a tolerance Ï. The gate is trained to distinguish safe from unsafe inputs, and a routing threshold is chosen via Clopper-Pearson conformal calibration on a held-out set, guaranteeing that the routed-set violation rate is at most α with probability 1-δ. We derive a feasibility condition linking safe routing to the base safe rate Ï and risk budget α, along with sufficient AUC thresholds ensuring that feasible routing exists. Across 35 OpenML datasets and multiple black-box model families, gate-based conformal routing maintains controlled violation while achieving substantially higher coverage than regression conformal and naive baselines. We further show that probabilistic calibration primarily affects routing efficiency rather than distribution-free validity.