Compound Poisson Processes, Latent Shrinkage Priors and Bayesian Nonconvex Penalization
This work addresses sparse learning for high-dimensional data analysis, presenting an incremental advancement by applying existing compound Poisson processes in a novel hierarchical Bayesian framework.
The paper tackles sparse learning problems by developing Bayesian nonconvex penalization methods using compound Poisson processes as latent shrinkage priors, and demonstrates feasibility and effectiveness in high-dimensional data analysis through empirical evaluation on simulated data.
In this paper we discuss Bayesian nonconvex penalization for sparse learning problems. We explore a nonparametric formulation for latent shrinkage parameters using subordinators which are one-dimensional Lévy processes. We particularly study a family of continuous compound Poisson subordinators and a family of discrete compound Poisson subordinators. We exemplify four specific subordinators: Gamma, Poisson, negative binomial and squared Bessel subordinators. The Laplace exponents of the subordinators are Bernstein functions, so they can be used as sparsity-inducing nonconvex penalty functions. We exploit these subordinators in regression problems, yielding a hierarchical model with multiple regularization parameters. We devise ECME (Expectation/Conditional Maximization Either) algorithms to simultaneously estimate regression coefficients and regularization parameters. The empirical evaluation of simulated data shows that our approach is feasible and effective in high-dimensional data analysis.