Constructive Disintegration and Conditional Modes
This addresses foundational mathematical issues in Bayesian statistics and machine learning, with implications for approximate Bayesian inference and inverse problems, though it appears incremental in clarifying existing concepts.
The paper tackles the difficulty of constructing disintegrations (the formalization of conditioning in Bayesian statistics) by providing mathematical tools to build them on differentiable manifolds, revealing a simple example where restricted densities and disintegration densities disagree. It also shows that conditional modes typically align with restricted measures rather than disintegration measures, with practical implications for Bayesian inference.
Conditioning, the central operation in Bayesian statistics, is formalised by the notion of disintegration of measures. However, due to the implicit nature of their definition, constructing disintegrations is often difficult. A folklore result in machine learning conflates the construction of a disintegration with the restriction of probability density functions onto the subset of events that are consistent with a given observation. We provide a comprehensive set of mathematical tools which can be used to construct disintegrations and apply these to find densities of disintegrations on differentiable manifolds. Using our results, we provide a disturbingly simple example in which the restricted density and the disintegration density drastically disagree. Motivated by applications in approximate Bayesian inference and Bayesian inverse problems, we further study the modes of disintegrations. We show that the recently introduced notion of a "conditional mode" does not coincide in general with the modes of the conditional measure obtained through disintegration, but rather the modes of the restricted measure. We also discuss the implications of the discrepancy between the two measures in practice, advocating for the utility of both approaches depending on the modelling context.