Information Theoretic Learning with Infinitely Divisible Kernels
This work addresses information theoretic learning for machine learning practitioners, offering a novel matrix-based approach that is incremental in its application to metric learning.
The authors tackled the problem of information theoretic learning by developing a framework based on infinitely divisible matrices, formulating an entropy-like functional from Renyi's axioms to avoid density estimation and leverage kernel spaces. They applied this to supervised metric learning, achieving results comparable to state-of-the-art methods.
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices. We formulate an entropy-like functional on positive definite matrices based on Renyi's axiomatic definition of entropy and examine some key properties of this functional that lead to the concept of infinite divisibility. The proposed formulation avoids the plug in estimation of density and brings along the representation power of reproducing kernel Hilbert spaces. As an application example, we derive a supervised metric learning algorithm using a matrix based analogue to conditional entropy achieving results comparable with the state of the art.