Neural-g: A Deep Learning Framework for Mixing Density Estimation
This work addresses a key bottleneck in empirical Bayes inference for statisticians and machine learning practitioners, offering a flexible tool for prior estimation, though it is incremental as it builds on neural network approaches.
The paper tackles the problem of mixing density estimation in empirical Bayes g-modeling by proposing neural-g, a neural network-based estimator that ensures valid probability densities and captures complex prior shapes like flat regions, heavy tails, and discontinuities, outperforming existing methods in simulations and real datasets.
Mixing (or prior) density estimation is an important problem in machine learning and statistics, especially in empirical Bayes $g$-modeling where accurately estimating the prior is necessary for making good posterior inferences. In this paper, we propose neural-$g$, a new neural network-based estimator for $g$-modeling. Neural-$g$ uses a softmax output layer to ensure that the estimated prior is a valid probability density. Under default hyperparameters, we show that neural-$g$ is very flexible and capable of capturing many unknown densities, including those with flat regions, heavy tails, and/or discontinuities. In contrast, existing methods struggle to capture all of these prior shapes. We provide justification for neural-$g$ by establishing a new universal approximation theorem regarding the capability of neural networks to learn arbitrary probability mass functions. To accelerate convergence of our numerical implementation, we utilize a weighted average gradient descent approach to update the network parameters. Finally, we extend neural-$g$ to multivariate prior density estimation. We illustrate the efficacy of our approach through simulations and analyses of real datasets. A software package to implement neural-$g$ is publicly available at https://github.com/shijiew97/neuralG.