Orthogonal Polynomials Approximation Algorithm (OPAA):a functional analytic approach to estimating probability densities
This method offers an alternative to optimization-based techniques for Bayesian evidence estimation, potentially benefiting researchers in statistics and machine learning, though it appears incremental as it builds on existing functional analytic concepts.
The paper tackles the problem of estimating probability densities and normalizing weights (evidence) by introducing the Orthogonal Polynomials Approximation Algorithm (OPAA), which uses a functional analytic approach to provide smooth estimates and parallelizable computations in one pass.
We present the new Orthogonal Polynomials Approximation Algorithm (OPAA), a parallelizable algorithm that estimates probability distributions using functional analytic approach: first, it finds a smooth functional estimate of the probability distribution, whether it is normalized or not; second, the algorithm provides an estimate of the normalizing weight; and third, the algorithm proposes a new computation scheme to compute such estimates. A core component of OPAA is a special transform of the square root of the joint distribution into a special functional space of our construct. Through this transform, the evidence is equated with the $L^2$ norm of the transformed function, squared. Hence, the evidence can be estimated by the sum of squares of the transform coefficients. Computations can be parallelized and completed in one pass. OPAA can be applied broadly to the estimation of probability density functions. In Bayesian problems, it can be applied to estimating the normalizing weight of the posterior, which is also known as the evidence, serving as an alternative to existing optimization-based methods.