Sparse Kernel Canonical Correlation Analysis via $\ell_1$-regularization
This work addresses a limitation in kernel CCA for researchers and practitioners needing sparse, interpretable nonlinear relationships between datasets, though it is incremental as it builds on existing kernel CCA methods.
The paper tackles the lack of sparsity in kernel canonical correlation analysis (CCA) by proposing a novel sparse kernel CCA algorithm (SKCCA) that introduces sparsity via ℓ₁-regularization, resulting in improved performance in computing sparse dual transformations and alleviating over-fitting.
Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between datasets. Despite their wide usage, they have one common limitation that is the lack of sparsity in their solution. In this paper, we consider sparse kernel CCA and propose a novel sparse kernel CCA algorithm (SKCCA). Our algorithm is based on a relationship between kernel CCA and least squares. Sparsity of the dual transformations is introduced by penalizing the $\ell_{1}$-norm of dual vectors. Experiments demonstrate that our algorithm not only performs well in computing sparse dual transformations but also can alleviate the over-fitting problem of kernel CCA.