TL-PCA: Transfer Learning of Principal Component Analysis
This addresses the issue of data scarcity in PCA for researchers and practitioners in machine learning, though it appears incremental as it builds on existing PCA methods with transfer learning.
The authors tackled the problem of PCA's performance being limited by scarce target data by proposing TL-PCA, a transfer learning approach that incorporates knowledge from a related source task, resulting in improved representation for test data in image datasets across various subspace dimensions.
Principal component analysis (PCA) can be significantly limited when there is too few examples of the target data of interest. We propose a transfer learning approach to PCA (TL-PCA) where knowledge from a related source task is used in addition to the scarce data of a target task. Our TL-PCA has two versions, one that uses a pretrained PCA solution of the source task, and another that uses the source data. Our proposed approach extends the PCA optimization objective with a penalty on the proximity of the target subspace and the source subspace as given by the pretrained source model or the source data. This optimization is solved by eigendecomposition for which the number of data-dependent eigenvectors (i.e., principal directions of TL-PCA) is not limited to the number of target data examples, which is a root cause that limits the standard PCA performance. Accordingly, our results for image datasets show that the representation of test data is improved by TL-PCA for dimensionality reduction where the learned subspace dimension is lower or higher than the number of target data examples.