uGMM-NN: Univariate Gaussian Mixture Model Neural Network
This work addresses the need for uncertainty-aware neural architectures in machine learning, offering a foundation for integrating probabilistic components, though it appears incremental as it builds on existing neural network frameworks.
The paper tackled the problem of embedding probabilistic reasoning into neural networks by introducing uGMM-NN, which parameterizes neuron activations as univariate Gaussian mixtures, achieving competitive discriminative performance compared to conventional multilayer perceptrons.
This paper introduces the Univariate Gaussian Mixture Model Neural Network (uGMM-NN), a novel neural architecture that embeds probabilistic reasoning directly into the computational units of deep networks. Unlike traditional neurons, which apply weighted sums followed by fixed nonlinearities, each uGMM-NN node parameterizes its activations as a univariate Gaussian mixture, with learnable means, variances, and mixing coefficients. This design enables richer representations by capturing multimodality and uncertainty at the level of individual neurons, while retaining the scalability of standard feedforward networks. We demonstrate that uGMM-NN can achieve competitive discriminative performance compared to conventional multilayer perceptrons, while additionally offering a probabilistic interpretation of activations. The proposed framework provides a foundation for integrating uncertainty-aware components into modern neural architectures, opening new directions for both discriminative and generative modeling.