A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
This work addresses a foundational limitation in kernel methods for machine learning by providing a more rigorous theoretical framework for conditional embeddings, though it appears incremental as it builds on existing unconditional embedding concepts.
The authors tackled the problem of conditional mean embeddings (CMEs) in reproducing kernel Hilbert spaces, which previously relied on restrictive operator-based assumptions, by developing a measure-theoretic approach that defines CMEs as random variables. They derived consistent empirical estimates with a regression interpretation and extended this to conditional versions of maximum mean discrepancy and Hilbert-Schmidt independence criteria, demonstrating performance through simulations.
We present an operator-free, measure-theoretic approach to the conditional mean embedding (CME) as a random variable taking values in a reproducing kernel Hilbert space. While the kernel mean embedding of unconditional distributions has been defined rigorously, the existing operator-based approach of the conditional version depends on stringent assumptions that hinder its analysis. We overcome this limitation via a measure-theoretic treatment of CMEs. We derive a natural regression interpretation to obtain empirical estimates, and provide a thorough theoretical analysis thereof, including universal consistency. As natural by-products, we obtain the conditional analogues of the maximum mean discrepancy and Hilbert-Schmidt independence criterion, and demonstrate their behaviour via simulations.