Fast Automatic Smoothing for Generalized Additive Models
This addresses a bottleneck in statistical modeling for researchers and practitioners using GAMs, offering incremental improvements in speed and reliability for large-scale applications.
The paper tackles the lack of reliable and fast methods for automatic smoothing in generalized additive models (GAMs) for large datasets by developing an empirical Bayes approach to learn optimal L2 regularization, resulting in an algorithm that is numerically stable, faster than existing methods, and achieves state-of-the-art accuracy.
Multiple generalized additive models (GAMs) are a type of distributional regression wherein parameters of probability distributions depend on predictors through smooth functions, with selection of the degree of smoothness via $L_2$ regularization. Multiple GAMs allow finer statistical inference by incorporating explanatory information in any or all of the parameters of the distribution. Owing to their nonlinearity, flexibility and interpretability, GAMs are widely used, but reliable and fast methods for automatic smoothing in large datasets are still lacking, despite recent advances. We develop a general methodology for automatically learning the optimal degree of $L_2$ regularization for multiple GAMs using an empirical Bayes approach. The smooth functions are penalized by different amounts, which are learned simultaneously by maximization of a marginal likelihood through an approximate expectation-maximization algorithm that involves a double Laplace approximation at the E-step, and leads to an efficient M-step. Empirical analysis shows that the resulting algorithm is numerically stable, faster than all existing methods and achieves state-of-the-art accuracy. For illustration, we apply it to an important and challenging problem in the analysis of extremal data.