MOSS: Multi-Objective Optimization for Stable Rule Sets
This work addresses the need for interpretable and stable rule-based models in machine learning, which is incremental as it builds on existing rule ensemble methods.
The authors tackled the problem of constructing stable and interpretable decision rule sets by developing MOSS, a multi-objective optimization framework that balances sparsity, accuracy, and stability, resulting in improved predictive performance and stability compared to state-of-the-art rule ensembles.
We present MOSS, a multi-objective optimization framework for constructing stable sets of decision rules. MOSS incorporates three important criteria for interpretability: sparsity, accuracy, and stability, into a single multi-objective optimization framework. Importantly, MOSS allows a practitioner to rapidly evaluate the trade-off between accuracy and stability in sparse rule sets in order to select an appropriate model. We develop a specialized cutting plane algorithm in our framework to rapidly compute the Pareto frontier between these two objectives, and our algorithm scales to problem instances beyond the capabilities of commercial optimization solvers. Our experiments show that MOSS outperforms state-of-the-art rule ensembles in terms of both predictive performance and stability.