Sparse Matrix-based Random Projection for Classification
This work offers an incremental improvement for machine learning practitioners by enhancing random projection techniques in classification tasks.
The paper tackled the problem of constructing random matrices for classification by focusing on feature selection rather than distance preservation, finding that sparse matrices with one nonzero element per column outperform denser ones when projection dimensions are large, achieving significant improvements in complexity and performance.
As a typical dimensionality reduction technique, random projection can be simply implemented with linear projection, while maintaining the pairwise distances of high-dimensional data with high probability. Considering this technique is mainly exploited for the task of classification, this paper is developed to study the construction of random matrix from the viewpoint of feature selection, rather than of traditional distance preservation. This yields a somewhat surprising theoretical result, that is, the sparse random matrix with exactly one nonzero element per column, can present better feature selection performance than other more dense matrices, if the projection dimension is sufficiently large (namely, not much smaller than the number of feature elements); otherwise, it will perform comparably to others. For random projection, this theoretical result implies considerable improvement on both complexity and performance, which is widely confirmed with the classification experiments on both synthetic data and real data.