Estimation and Feature Selection in Mixtures of Generalized Linear Experts Models
This addresses the problem of modeling heterogeneous data with high-dimensional predictors for researchers in statistical learning, though it appears incremental as it builds on existing MoE frameworks.
The paper tackles the challenge of parameter estimation and feature selection in high-dimensional Mixtures-of-Experts models by proposing a regularized maximum likelihood estimation with a proximal-Newton EM algorithm, showing good performance in recovering sparse solutions and clustering compared to state-of-the-art methods.
Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification, as well as for clustering. Their estimation in high-dimensional problems is still however challenging. We consider the problem of parameter estimation and feature selection in MoE models with different generalized linear experts models, and propose a regularized maximum likelihood estimation that efficiently encourages sparse solutions for heterogeneous data with high-dimensional predictors. The developed proximal-Newton EM algorithm includes proximal Newton-type procedures to update the model parameter by monotonically maximizing the objective function and allows to perform efficient estimation and feature selection. An experimental study shows the good performance of the algorithms in terms of recovering the actual sparse solutions, parameter estimation, and clustering of heterogeneous regression data, compared to the main state-of-the art competitors.