A Family of Kernelized Matrix Costs for Multiple-Output Mixture Neural Networks
This work addresses density estimation in generative models, but appears incremental as it extends existing methods with new cost functions.
The paper tackled the problem of data density approximation by combining Mixture Density Networks with contrastive costs, proposing four kernelized matrix costs in Hilbert space, and achieved learning multiple centers for mixture densities.
Pairwise distance-based costs are crucial for self-supervised and contrastive feature learning. Mixture Density Networks (MDNs) are a widely used approach for generative models and density approximation, using neural networks to produce multiple centers that define a Gaussian mixture. By combining MDNs with contrastive costs, this paper proposes data density approximation using four types of kernelized matrix costs in the Hilbert space: the scalar cost, the vector-matrix cost, the matrix-matrix cost (the trace of Schur complement), and the SVD cost (the nuclear norm), for learning multiple centers required to define a mixture density.