Lasso for hierarchical polynomial models
This work addresses the limitation of existing methods that only handle degree-two models, offering an incremental improvement for statistical modeling in regression analysis.
The authors tackled the problem of estimating hierarchical polynomial regression models beyond degree two by deriving constraints for model parameters based on polynomial hierarchy, and they applied lasso with quadratic programming to achieve lower validation error and smaller model size compared to existing techniques.
In a polynomial regression model, the divisibility conditions implicit in polynomial hierarchy give way to a natural construction of constraints for the model parameters. We use this principle to derive versions of strong and weak hierarchy and to extend existing work in the literature, which at the moment is only concerned with models of degree two. We discuss how to estimate parameters in lasso using standard quadratic programming techniques and apply our proposal to both simulated data and examples from the literature. The proposed methodology compares favorably with existing techniques in terms of low validation error and model size.