The Jacobian and Hessian of the Kullback-Leibler Divergence between Multivariate Gaussian Distributions (Technical Report)
This provides a technical reference for researchers in statistics and machine learning, but it is incremental as it builds on established methods without introducing new paradigms.
The paper tackles the problem of deriving the Jacobian and Hessian matrices for the Kullback-Leibler divergence between multivariate Gaussian distributions, presenting detailed derivations based on differential calculus and existing theoretical frameworks.
This document shows how to obtain the Jacobian and Hessian matrices of the Kullback-Leibler divergence between two multivariate Gaussian distributions, using the first and second-order differentials. The presented derivations are based on the theory presented by \cite{magnus99}. I've also got great inspiration from some of the derivations in \cite{minka}. Since I pretend to be at most didactic, the document is split into a summary of results and detailed derivations on each of the elements involved, with specific references to the tricks used in the derivations, and to many of the underlying concepts.