Dynamic Model Selection for Prediction Under a Budget
This addresses the problem of efficient prediction under budget constraints for machine learning practitioners, though it is incremental as it builds on existing model selection and gating techniques.
The paper tackles the problem of resource-constrained prediction by developing a dynamic model selection approach that uses a gating function to choose among models at test-time, aiming to minimize cost without sacrificing accuracy; it outperforms state-of-the-art methods on benchmark datasets by achieving higher accuracy for the same cost.
We present a dynamic model selection approach for resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gating and prediction models on fully labeled training data by means of a bottom-up strategy. Our novel bottom-up method is a recursive scheme whereby a high-accuracy complex model is first trained. Then a low-complexity gating and prediction model are subsequently learnt to adaptively approximate the high-accuracy model in regions where low-cost models are capable of making highly accurate predictions. We pose an empirical loss minimization problem with cost constraints to jointly train gating and prediction models. On a number of benchmark datasets our method outperforms state-of-the-art achieving higher accuracy for the same cost.