Supervised PCA: A Multiobjective Approach
This work addresses the need for more effective feature extraction in supervised learning tasks, though it is incremental as it builds on prior SPCA methods.
The authors tackled the problem of supervised principal component analysis (SPCA) by proposing a multiobjective method that jointly optimizes prediction error and variance explained, outperforming existing approaches in both metrics.
Methods for supervised principal component analysis (SPCA) aim to incorporate label information into principal component analysis (PCA), so that the extracted features are more useful for a prediction task of interest. Prior work on SPCA has focused primarily on optimizing prediction error, and has neglected the value of maximizing variance explained by the extracted features. We propose a new method for SPCA that addresses both of these objectives jointly, and demonstrate empirically that our approach dominates existing approaches, i.e., outperforms them with respect to both prediction error and variation explained. Our approach accommodates arbitrary supervised learning losses and, through a statistical reformulation, provides a novel low-rank extension of generalized linear models.