Differentiable Model Selection for Ensemble Learning
This work addresses model selection for ensemble learning, which is an incremental improvement in optimizing ensemble performance for machine learning practitioners.
The paper tackles the challenge of selecting optimal models for each input sample in ensemble learning by proposing a differentiable model selection framework that integrates machine learning and combinatorial optimization. The framework outperforms conventional and advanced consensus rules across various tasks, demonstrating versatility and effectiveness.
Model selection is a strategy aimed at creating accurate and robust models. A key challenge in designing these algorithms is identifying the optimal model for classifying any particular input sample. This paper addresses this challenge and proposes a novel framework for differentiable model selection integrating machine learning and combinatorial optimization. The framework is tailored for ensemble learning, a strategy that combines the outputs of individually pre-trained models, and learns to select appropriate ensemble members for a particular input sample by transforming the ensemble learning task into a differentiable selection program trained end-to-end within the ensemble learning model. Tested on various tasks, the proposed framework demonstrates its versatility and effectiveness, outperforming conventional and advanced consensus rules across a variety of settings and learning tasks.