Latent Gaussian Model Boosting
This work addresses prediction accuracy issues for users of statistical and machine learning models, but it is incremental as it builds on existing techniques.
The paper tackles the limitations of tree-boosting (e.g., conditional independence assumptions, discontinuous predictions) and latent Gaussian models (e.g., unrealistic zero or linear prior mean functions) by combining them into a novel approach, resulting in increased prediction accuracy in simulated and real-world data experiments.
Latent Gaussian models and boosting are widely used techniques in statistics and machine learning. Tree-boosting shows excellent prediction accuracy on many data sets, but potential drawbacks are that it assumes conditional independence of samples, produces discontinuous predictions for, e.g., spatial data, and it can have difficulty with high-cardinality categorical variables. Latent Gaussian models, such as Gaussian process and grouped random effects models, are flexible prior models which explicitly model dependence among samples and which allow for efficient learning of predictor functions and for making probabilistic predictions. However, existing latent Gaussian models usually assume either a zero or a linear prior mean function which can be an unrealistic assumption. This article introduces a novel approach that combines boosting and latent Gaussian models to remedy the above-mentioned drawbacks and to leverage the advantages of both techniques. We obtain increased prediction accuracy compared to existing approaches in both simulated and real-world data experiments.