A Discriminative Gaussian Mixture Model with Sparsity
This work addresses a specific problem in probabilistic classification for researchers and practitioners by offering an incremental improvement through sparsity and integration into neural networks.
The authors tackled the limitation of softmax-based discriminative models, which assume unimodality per class, by proposing a sparse discriminative Gaussian mixture model (SDGM) that reduces parameters and improves generalization, with experimental results showing it outperforms existing models.
In probabilistic classification, a discriminative model based on the softmax function has a potential limitation in that it assumes unimodality for each class in the feature space. The mixture model can address this issue, although it leads to an increase in the number of parameters. We propose a sparse classifier based on a discriminative GMM, referred to as a sparse discriminative Gaussian mixture (SDGM). In the SDGM, a GMM-based discriminative model is trained via sparse Bayesian learning. Using this sparse learning framework, we can simultaneously remove redundant Gaussian components and reduce the number of parameters used in the remaining components during learning; this learning method reduces the model complexity, thereby improving the generalization capability. Furthermore, the SDGM can be embedded into neural networks (NNs), such as convolutional NNs, and can be trained in an end-to-end manner. Experimental results demonstrated that the proposed method outperformed the existing softmax-based discriminative models.