Functional Mixtures-of-Experts
This work addresses prediction with heterogeneous functional data, such as time series, for domains like statistics and machine learning, but it is incremental as it extends existing Mixtures-of-Experts frameworks to functional contexts.
The authors tackled the problem of predicting heterogeneous data that includes functional observations like time series by extending Mixtures-of-Experts models to functional data analysis, resulting in functional ME (FME) and sparse iFME models that accurately capture complex nonlinear relationships and cluster data in simulations and real applications.
We consider the statistical analysis of heterogeneous data for prediction in situations where the observations include functions, typically time series. We extend the modeling with Mixtures-of-Experts (ME), as a framework of choice in modeling heterogeneity in data for prediction with vectorial observations, to this functional data analysis context. We first present a new family of ME models, named functional ME (FME) in which the predictors are potentially noisy observations, from entire functions. Furthermore, the data generating process of the predictor and the real response, is governed by a hidden discrete variable representing an unknown partition. Second, by imposing sparsity on derivatives of the underlying functional parameters via Lasso-like regularizations, we provide sparse and interpretable functional representations of the FME models called iFME. We develop dedicated expectation--maximization algorithms for Lasso-like (EM-Lasso) regularized maximum-likelihood parameter estimation strategies to fit the models. The proposed models and algorithms are studied in simulated scenarios and in applications to two real data sets, and the obtained results demonstrate their performance in accurately capturing complex nonlinear relationships and in clustering the heterogeneous regression data.