Regularization and feature selection for large dimensional data
This work addresses the curse of dimensionality in domains like bioinformatics and text classification, but it is incremental as it compares existing regularization-based methods.
The paper tackled feature selection for high-dimensional data by evaluating five embedded methods using ridge and Lasso regularization, comparing them on sparsity, correlation, and execution times across five datasets.
Feature selection has evolved to be an important step in several machine learning paradigms. In domains like bio-informatics and text classification which involve data of high dimensions, feature selection can help in drastically reducing the feature space. In cases where it is difficult or infeasible to obtain sufficient number of training examples, feature selection helps overcome the curse of dimensionality which in turn helps improve performance of the classification algorithm. The focus of our research here are five embedded feature selection methods which use either the ridge regression, or Lasso regression, or a combination of the two in the regularization part of the optimization function. We evaluate five chosen methods on five large dimensional datasets and compare them on the parameters of sparsity and correlation in the datasets and their execution times.