RULLS: Randomized Union of Locally Linear Subspaces for Feature Engineering
This addresses the challenge of data preparation and representation selection in machine learning, offering a robust feature engineering solution, though it appears incremental as it builds on existing subspace-based approaches.
The paper tackled the problem of feature engineering by proposing RULLS, a method that generates sparse, non-negative, and rotation invariant features in an unsupervised manner, and demonstrated its effectiveness on real-world datasets for clustering and classification tasks with high performance compared to existing methods.
Feature engineering plays an important role in the success of a machine learning model. Most of the effort in training a model goes into data preparation and choosing the right representation. In this paper, we propose a robust feature engineering method, Randomized Union of Locally Linear Subspaces (RULLS). We generate sparse, non-negative, and rotation invariant features in an unsupervised fashion. RULLS aggregates features from a random union of subspaces by describing each point using globally chosen landmarks. These landmarks serve as anchor points for choosing subspaces. Our method provides a way to select features that are relevant in the neighborhood around these chosen landmarks. Distances from each data point to $k$ closest landmarks are encoded in the feature matrix. The final feature representation is a union of features from all chosen subspaces. The effectiveness of our algorithm is shown on various real-world datasets for tasks such as clustering and classification of raw data and in the presence of noise. We compare our method with existing feature generation methods. Results show a high performance of our method on both classification and clustering tasks.