A Rigorous Theory of Conditional Mean Embeddings
This provides a foundational improvement for researchers using kernel methods in machine learning, though it is incremental as it builds on existing CME concepts.
The paper tackles the lack of a rigorous mathematical foundation for conditional mean embeddings (CMEs) by developing a theory for both centred and uncentred variants, weakening applicability conditions and revealing a connection to Gaussian conditioning in Hilbert spaces.
Conditional mean embeddings (CMEs) have proven themselves to be a powerful tool in many machine learning applications. They allow the efficient conditioning of probability distributions within the corresponding reproducing kernel Hilbert spaces (RKHSs) by providing a linear-algebraic relation for the kernel mean embeddings of the respective joint and conditional probability distributions. Both centred and uncentred covariance operators have been used to define CMEs in the existing literature. In this paper, we develop a mathematically rigorous theory for both variants, discuss the merits and problems of each, and significantly weaken the conditions for applicability of CMEs. In the course of this, we demonstrate a beautiful connection to Gaussian conditioning in Hilbert spaces.