Deep Unsupervised Feature Selection by Discarding Nuisance and Correlated Features
This addresses the challenge of handling noisy and redundant features in high-dimensional data for machine learning practitioners, representing an incremental improvement over methods that handle only one type of feature.
The paper tackles the problem of selecting features in unsupervised learning by discarding both nuisance and correlated features, proposing a differentiable method that uses an autoencoder and Laplacian score to outperform existing approaches on real-world datasets, achieving state-of-the-art clustering results.
Modern datasets often contain large subsets of correlated features and nuisance features, which are not or loosely related to the main underlying structures of the data. Nuisance features can be identified using the Laplacian score criterion, which evaluates the importance of a given feature via its consistency with the Graph Laplacians' leading eigenvectors. We demonstrate that in the presence of large numbers of nuisance features, the Laplacian must be computed on the subset of selected features rather than on the complete feature set. To do this, we propose a fully differentiable approach for unsupervised feature selection, utilizing the Laplacian score criterion to avoid the selection of nuisance features. We employ an autoencoder architecture to cope with correlated features, trained to reconstruct the data from the subset of selected features. Building on the recently proposed concrete layer that allows controlling for the number of selected features via architectural design, simplifying the optimization process. Experimenting on several real-world datasets, we demonstrate that our proposed approach outperforms similar approaches designed to avoid only correlated or nuisance features, but not both. Several state-of-the-art clustering results are reported.