Class Mean Vector Component and Discriminant Analysis
This work addresses a crucial issue in kernel-based methods for researchers in machine learning, but it appears incremental as it builds on existing techniques like kernel PCA and discriminant analysis.
The paper tackles the problem of selecting an optimal kernel subspace for dimensionality reduction by proposing a method that preserves pairwise distances of class means in the feature space, with analysis linking it to kernel principal component analysis and kernel discriminant analysis.
The kernel matrix used in kernel methods encodes all the information required for solving complex nonlinear problems defined on data representations in the input space using simple, but implicitly defined, solutions. Spectral analysis on the kernel matrix defines an explicit nonlinear mapping of the input data representations to a subspace of the kernel space, which can be used for directly applying linear methods. However, the selection of the kernel subspace is crucial for the performance of the proceeding processing steps. In this paper, we propose a component analysis method for kernel-based dimensionality reduction that optimally preserves the pair-wise distances of the class means in the feature space. We provide extensive analysis on the connection of the proposed criterion to those used in kernel principal component analysis and kernel discriminant analysis, leading to a discriminant analysis version of the proposed method. Our analysis also provides more insights on the properties of the feature spaces obtained by applying these methods.