Principal Model Analysis Based on Partial Least Squares
This work addresses a domain-specific problem in machine learning for classification tasks, presenting an incremental improvement over existing dimension reduction techniques.
The paper tackles the problem of improving classification performance and stability in dimension reduction by proposing a Principal Model Analysis (PMA) method that combines PCA and PLS. Experimental results on six public datasets show that PMA achieves better classification performance and is usually more stable compared to traditional methods like PLS and LDA.
Motivated by the Bagging Partial Least Squares (PLS) and Principal Component Analysis (PCA) algorithms, we propose a Principal Model Analysis (PMA) method in this paper. In the proposed PMA algorithm, the PCA and the PLS are combined. In the method, multiple PLS models are trained on sub-training sets, derived from the original training set based on the random sampling with replacement method. The regression coefficients of all the sub-PLS models are fused in a joint regression coefficient matrix. The final projection direction is then estimated by performing the PCA on the joint regression coefficient matrix. The proposed PMA method is compared with other traditional dimension reduction methods, such as PLS, Bagging PLS, Linear discriminant analysis (LDA) and PLS-LDA. Experimental results on six public datasets show that our proposed method can achieve better classification performance and is usually more stable.