Gaussian Gated Linear Networks
This work addresses the need for efficient and interpretable deep learning models in regression tasks, though it is incremental as it builds on the existing GLN framework.
The authors tackled the problem of extending deep neural networks to regression and density modeling without backpropagation by proposing the Gaussian Gated Linear Network (G-GLN), which achieved competitive or state-of-the-art performance on multiple benchmarks.
We propose the Gaussian Gated Linear Network (G-GLN), an extension to the recently proposed GLN family of deep neural networks. Instead of using backpropagation to learn features, GLNs have a distributed and local credit assignment mechanism based on optimizing a convex objective. This gives rise to many desirable properties including universality, data-efficient online learning, trivial interpretability and robustness to catastrophic forgetting. We extend the GLN framework from classification to multiple regression and density modelling by generalizing geometric mixing to a product of Gaussian densities. The G-GLN achieves competitive or state-of-the-art performance on several univariate and multivariate regression benchmarks, and we demonstrate its applicability to practical tasks including online contextual bandits and density estimation via denoising.