Disentangling Interpretable Factors with Supervised Independent Subspace Principal Component Analysis
This work addresses the need for explainable representation learning in high-dimensional data analysis, particularly for domains like healthcare and biology, though it is incremental as it builds on PCA and existing subspace methods.
The paper tackled the problem of creating interpretable representations for high-dimensional data by introducing Supervised Independent Subspace Principal Component Analysis (sisPCA), which incorporates supervision and ensures subspace disentanglement, demonstrating its effectiveness in applications like breast cancer diagnosis and malaria infection analysis to reveal distinct functional pathways.
The success of machine learning models relies heavily on effectively representing high-dimensional data. However, ensuring data representations capture human-understandable concepts remains difficult, often requiring the incorporation of prior knowledge and decomposition of data into multiple subspaces. Traditional linear methods fall short in modeling more than one space, while more expressive deep learning approaches lack interpretability. Here, we introduce Supervised Independent Subspace Principal Component Analysis ($\texttt{sisPCA}$), a PCA extension designed for multi-subspace learning. Leveraging the Hilbert-Schmidt Independence Criterion (HSIC), $\texttt{sisPCA}$ incorporates supervision and simultaneously ensures subspace disentanglement. We demonstrate $\texttt{sisPCA}$'s connections with autoencoders and regularized linear regression and showcase its ability to identify and separate hidden data structures through extensive applications, including breast cancer diagnosis from image features, learning aging-associated DNA methylation changes, and single-cell analysis of malaria infection. Our results reveal distinct functional pathways associated with malaria colonization, underscoring the essentiality of explainable representation in high-dimensional data analysis.