Penalty, Shrinkage, and Preliminary Test Estimators under Full Model Hypothesis
This work provides incremental insights for statisticians and data scientists by clarifying the relative performance of popular estimators in regression analysis.
The paper compares penalty estimators (ridge regression, LASSO, adaptive LASSO, SCAD, elastic net) against traditional estimators (Least Squares, restricted, preliminary test, Stein-type) in multiple regression under full model hypothesis, finding that ridge regression uniformly dominates all traditional estimators, while LASSO and others only dominate Least Squares.
This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimator, preliminary test estimator, and Stein-type estimators when the dimension of the parameter space is smaller than the sample space dimension. We find that RR uniformly dominates LSE, RE, PTE, SE and PRSE while LASSO, aLASSO, SCAD, and EN uniformly dominates LSE only. Further, it is observed that neither penalty estimators nor Stein-type estimator dominate one another.