Kernel Density Matrices for Probabilistic Deep Learning
This work addresses the problem of probabilistic modeling in machine learning for researchers and practitioners, offering a novel representation that integrates into deep neural networks, though it appears incremental as it builds on existing density matrix concepts.
The paper tackles the challenge of representing joint probability distributions for both continuous and discrete variables in probabilistic deep learning by introducing kernel density matrices, a simpler and effective mechanism derived from quantum mechanics concepts. The result is a versatile framework that enables differentiable models for tasks like density estimation and generative modeling, with implementation available as an open-source library.
This paper introduces a novel approach to probabilistic deep learning, kernel density matrices, which provide a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables. In quantum mechanics, a density matrix is the most general way to describe the state of a quantum system. This work extends the concept of density matrices by allowing them to be defined in a reproducing kernel Hilbert space. This abstraction allows the construction of differentiable models for density estimation, inference, and sampling, and enables their integration into end-to-end deep neural models. In doing so, we provide a versatile representation of marginal and joint probability distributions that allows us to develop a differentiable, compositional, and reversible inference procedure that covers a wide range of machine learning tasks, including density estimation, discriminative learning, and generative modeling. The broad applicability of the framework is illustrated by two examples: an image classification model that can be naturally transformed into a conditional generative model, and a model for learning with label proportions that demonstrates the framework's ability to deal with uncertainty in the training samples. The framework is implemented as a library and is available at: https://github.com/fagonzalezo/kdm.