Deep Data Density Estimation through Donsker-Varadhan Representation
This addresses a fundamental issue in machine learning for researchers and practitioners, but it appears incremental as it builds on existing variational bounds.
The paper tackles the challenging problem of data density estimation in deep learning by proposing a method using deep neural networks and the Donsker-Varadhan variational lower bound on KL divergence, showing it is competitive with previous methods.
Estimating the data density is one of the challenging problems in deep learning. In this paper, we present a simple yet effective method for estimating the data density using a deep neural network and the Donsker-Varadhan variational lower bound on the KL divergence. We show that the optimal critic function associated with the Donsker-Varadhan representation on the KL divergence between the data and the uniform distribution can estimate the data density. We also present the deep neural network-based modeling and its stochastic learning. The experimental results and possible applications of the proposed method demonstrate that it is competitive with the previous methods and has a lot of possibilities in applied to various applications.