"A 6 or a 9?": Ensemble Learning Through the Multiplicity of Performant Models and Explanations
This addresses the problem of model generalization for applications like manufacturing and medical diagnosis, offering an incremental improvement over existing ensemble methods.
The paper tackles the challenge of selecting models that generalize well by proposing the Rashomon Ensemble, which strategically selects diverse high-performing models based on performance and explanations, resulting in up to 0.20+ AUROC improvements in scenarios with a large Rashomon ratio.
Creating models from past observations and ensuring their effectiveness on new data is the essence of machine learning. However, selecting models that generalize well remains a challenging task. Related to this topic, the Rashomon Effect refers to cases where multiple models perform similarly well for a given learning problem. This often occurs in real-world scenarios, like the manufacturing process or medical diagnosis, where diverse patterns in data lead to multiple high-performing solutions. We propose the Rashomon Ensemble, a method that strategically selects models from these diverse high-performing solutions to improve generalization. By grouping models based on both their performance and explanations, we construct ensembles that maximize diversity while maintaining predictive accuracy. This selection ensures that each model covers a distinct region of the solution space, making the ensemble more robust to distribution shifts and variations in unseen data. We validate our approach on both open and proprietary collaborative real-world datasets, demonstrating up to 0.20+ AUROC improvements in scenarios where the Rashomon ratio is large. Additionally, we demonstrate tangible benefits for businesses in various real-world applications, highlighting the robustness, practicality, and effectiveness of our approach.