Soft Rule Ensembles for Statistical Learning
This work addresses supervised learning challenges by proposing an incremental improvement to rule-based methods, potentially benefiting practitioners in statistical learning and data analysis.
The paper tackles supervised learning problems by developing soft rule ensembles, which are created by applying logistic regression with Firth's bias correction to hard rules generated from importance sampling learning ensembles. The result shows that soft rule ensembles improve predictive performance over hard rule ensembles, as demonstrated through various examples and simulations.
In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the perfect separation problem related to the logistic regression, Firth's bias corrected likelihood is used. Various examples and simulation results show that soft rule ensembles can improve predictive performance over hard rule ensembles.