Gaussian RBF Centered Kernel Alignment (CKA) in the Large Bandwidth Limit
arXiv:2112.09305v117 citations
Originality Synthesis-oriented
AI Analysis
This provides theoretical insights into kernel methods for representation analysis, but is incremental as it builds on existing CKA theory.
The paper proves that Gaussian RBF Centered Kernel Alignment (CKA) converges to linear CKA as bandwidth increases, with convergence onset influenced by feature geometry and bounded by representation eccentricity.
We prove that Centered Kernel Alignment (CKA) based on a Gaussian RBF kernel converges to linear CKA in the large-bandwidth limit. We show that convergence onset is sensitive to the geometry of the feature representations, and that representation eccentricity bounds the range of bandwidths for which Gaussian CKA behaves nonlinearly.