SplitWise Regression: Stepwise Modeling with Adaptive Dummy Encoding
This work addresses a persistent problem in regression modeling for researchers and practitioners seeking interpretable models, though it appears incremental as it builds on existing stepwise regression techniques.
The paper tackles the challenge of capturing nonlinear relationships while maintaining interpretability in regression modeling by introducing SplitWise, a framework that enhances stepwise regression with adaptive dummy encoding, resulting in more parsimonious and generalizable models compared to traditional methods.
Capturing nonlinear relationships without sacrificing interpretability remains a persistent challenge in regression modeling. We introduce SplitWise, a novel framework that enhances stepwise regression. It adaptively transforms numeric predictors into threshold-based binary features using shallow decision trees, but only when such transformations improve model fit, as assessed by the Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC). This approach preserves the transparency of linear models while flexibly capturing nonlinear effects. Implemented as a user-friendly R package, SplitWise is evaluated on both synthetic and real-world datasets. The results show that it consistently produces more parsimonious and generalizable models than traditional stepwise and penalized regression techniques.