Manifold Partition Discriminant Analysis
This work addresses the problem of improving supervised dimensionality reduction for machine learning practitioners by better capturing within-class similarity and inter-class separation, representing an incremental improvement over existing graph Laplacian methods.
The paper proposes Manifold Partition Discriminant Analysis (MPDA), a supervised dimensionality reduction algorithm. It aims to find a linear embedding where within-class similarity aligns with local manifold variation and different classes are separated. The method improves within-class similarity by representing the data manifold in a piecewise manner, leading to improved dimensionality reduction performance on real-world datasets.
We propose a novel algorithm for supervised dimensionality reduction named Manifold Partition Discriminant Analysis (MPDA). It aims to find a linear embedding space where the within-class similarity is achieved along the direction that is consistent with the local variation of the data manifold, while nearby data belonging to different classes are well separated. By partitioning the data manifold into a number of linear subspaces and utilizing the first-order Taylor expansion, MPDA explicitly parameterizes the connections of tangent spaces and represents the data manifold in a piecewise manner. While graph Laplacian methods capture only the pairwise interaction between data points, our method capture both pairwise and higher order interactions (using regional consistency) between data points. This manifold representation can help to improve the measure of within-class similarity, which further leads to improved performance of dimensionality reduction. Experimental results on multiple real-world data sets demonstrate the effectiveness of the proposed method.