Feature selection or extraction decision process for clustering using PCA and FRSD
This addresses a decision-making challenge for data scientists in unsupervised learning, but it is incremental as it combines existing techniques.
The paper tackles the problem of deciding between feature selection or extraction before clustering in unsupervised learning, proposing a method that uses FRSD, PCA, and K-Means with Silhouette Index, and tests it on 5 smart city dataset use cases.
This paper concerns the critical decision process of extracting or selecting the features before applying a clustering algorithm. It is not obvious to evaluate the importance of the features since the most popular methods to do it are usually made for a supervised learning technique process. A clustering algorithm is an unsupervised method. It means that there is no known output label to match the input data. This paper proposes a new method to choose the best dimensionality reduction method (selection or extraction) according to the data scientist's parameters, aiming to apply a clustering process at the end. It uses Feature Ranking Process Based on Silhouette Decomposition (FRSD) algorithm, a Principal Component Analysis (PCA) algorithm, and a K-Means algorithm along with its metric, the Silhouette Index (SI). This paper presents 5 use cases based on a smart city dataset. This research also aims to discuss the impacts, the advantages, and the disadvantages of each choice that can be made in this unsupervised learning process.