Characteristic and Universal Tensor Product Kernels
This work provides theoretical foundations for kernel-based statistical tests, which is incremental but important for researchers in machine learning and statistics.
The paper addresses the open questions of when the Hilbert-Schmidt independence criterion (HSIC) characterizes independence and when maximum mean discrepancy (MMD) with tensor product kernels can discriminate probability distributions, by studying the characteristic properties of tensor product kernels.
Maximum mean discrepancy (MMD), also called energy distance or N-distance in statistics and Hilbert-Schmidt independence criterion (HSIC), specifically distance covariance in statistics, are among the most popular and successful approaches to quantify the difference and independence of random variables, respectively. Thanks to their kernel-based foundations, MMD and HSIC are applicable on a wide variety of domains. Despite their tremendous success, quite little is known about when HSIC characterizes independence and when MMD with tensor product kernel can discriminate probability distributions. In this paper, we answer these questions by studying various notions of characteristic property of the tensor product kernel.