Particle swarm optimization with Applications to Maximum Likelihood Estimation and Penalized Negative Binomial Regression
This addresses optimization challenges for statisticians and data analysts working with complex models, though it is incremental as it applies an existing optimization method to statistical problems.
The paper tackles the problem of parameter estimation in nonstandard statistical distributions by proposing Particle Swarm Optimization (PSO) as an alternative to existing optimization routines like nlminb and optim, finding that PSO can reproduce or improve results, handle non-convergence cases, and identify issues in examples such as log-binomial regression and LASSO-penalized models.
General purpose optimization routines such as nlminb, optim (R) or nlmixed (SAS) are frequently used to estimate model parameters in nonstandard distributions. This paper presents Particle Swarm Optimization (PSO), as an alternative to many of the current algorithms used in statistics. We find that PSO can not only reproduce the same results as the above routines, it can also produce results that are more optimal or when others cannot converge. In the latter case, it can also identify the source of the problem or problems. We highlight advantages of using PSO using four examples, where: (1) some parameters in a generalized distribution are unidentified using PSO when it is not apparent or computationally manifested using routines in R or SAS; (2) PSO can produce estimation results for the log-binomial regressions when current routines may not; (3) PSO provides flexibility in the link function for binomial regression with LASSO penalty, which is unsupported by standard packages like GLM and GENMOD in Stata and SAS, respectively, and (4) PSO provides superior MLE estimates for an EE-IW distribution compared with those from the traditional statistical methods that rely on moments.