Training Neural Networks for Likelihood/Density Ratio Estimation
This work offers a data-driven approach for likelihood ratio estimation, which is incremental as it adapts classical methods to neural networks for engineering and statistics applications.
The authors tackled the problem of estimating likelihood/density ratios from data without assuming known densities, by developing neural network-based solutions for detection and hypothesis testing. They provided a unified methodology with guarantees that the optimization yields the correct function.
Various problems in Engineering and Statistics require the computation of the likelihood ratio function of two probability densities. In classical approaches the two densities are assumed known or to belong to some known parametric family. In a data-driven version we replace this requirement with the availability of data sampled from the densities of interest. For most well known problems in Detection and Hypothesis testing we develop solutions by providing neural network based estimates of the likelihood ratio or its transformations. This task necessitates the definition of proper optimizations which can be used for the training of the network. The main purpose of this work is to offer a simple and unified methodology for defining such optimization problems with guarantees that the solution is indeed the desired function. Our results are extended to cover estimates for likelihood ratios of conditional densities and estimates for statistics encountered in local approaches.