From explained variance of correlated components to PCA without orthogonality constraints
This work addresses a technical bottleneck in sparse PCA design for researchers in machine learning and statistics, but it is incremental as it builds on existing methods.
The paper tackles the difficulty of applying L1 regularization to Block PCA due to orthogonality constraints by introducing new objective functions to measure variance explained by correlated components, showing that only two of six definitions are suitable for PCA without orthogonality constraints.
Block Principal Component Analysis (Block PCA) of a data matrix A, where loadings Z are determined by maximization of AZ 2 over unit norm orthogonal loadings, is difficult to use for the design of sparse PCA by 1 regularization, due to the difficulty of taking care of both the orthogonality constraint on loadings and the non differentiable 1 penalty. Our objective in this paper is to relax the orthogonality constraint on loadings by introducing new objective functions expvar(Y) which measure the part of the variance of the data matrix A explained by correlated components Y = AZ. So we propose first a comprehensive study of mathematical and numerical properties of expvar(Y) for two existing definitions Zou et al. [2006], Shen and Huang [2008] and four new definitions. Then we show that only two of these explained variance are fit to use as objective function in block PCA formulations for A rid of orthogonality constraints.