The linear conditional expectation in Hilbert space
This work addresses foundational mathematical underpinnings for approximate Bayesian inference and kernel methods, offering incremental improvements in theoretical understanding.
The paper establishes analytical properties of the linear conditional expectation in infinite-dimensional Hilbert spaces and develops a regularization procedure, providing a simpler derivation and justification for the conditional mean embedding formula used in machine learning.
The linear conditional expectation (LCE) provides a best linear (or rather, affine) estimate of the conditional expectation and hence plays an important rôle in approximate Bayesian inference, especially the Bayes linear approach. This article establishes the analytical properties of the LCE in an infinite-dimensional Hilbert space context. In addition, working in the space of affine Hilbert--Schmidt operators, we establish a regularisation procedure for this LCE. As an important application, we obtain a simple alternative derivation and intuitive justification of the conditional mean embedding formula, a concept widely used in machine learning to perform the conditioning of random variables by embedding them into reproducing kernel Hilbert spaces.