Grouping effects of sparse CCA models in variable selection
This work addresses variable selection challenges in high-dimensional data analysis for researchers in statistics and machine learning, though it is incremental as it builds on existing SCCA models.
The paper analyzes the grouping effect of standard and simplified sparse canonical correlation analysis (SCCA) models in variable selection, finding that simplified SCCA jointly selects or deselects groups of correlated variables, while standard SCCA randomly picks dominant variables from each group, with empirical verification on synthetic and real imaging genetics data.
The sparse canonical correlation analysis (SCCA) is a bi-multivariate association model that finds sparse linear combinations of two sets of variables that are maximally correlated with each other. In addition to the standard SCCA model, a simplified SCCA criterion which maixmizes the cross-covariance between a pair of canonical variables instead of their cross-correlation, is widely used in the literature due to its computational simplicity. However, the behaviors/properties of the solutions of these two models remain unknown in theory. In this paper, we analyze the grouping effect of the standard and simplified SCCA models in variable selection. In high-dimensional settings, the variables often form groups with high within-group correlation and low between-group correlation. Our theoretical analysis shows that for grouped variable selection, the simplified SCCA jointly selects or deselects a group of variables together, while the standard SCCA randomly selects a few dominant variables from each relevant group of correlated variables. Empirical results on synthetic data and real imaging genetics data verify the finding of our theoretical analysis.