Feature Subset Weighting for Distance-based Supervised Learning through Choquet Integration
This work addresses the challenge of enhancing distance metrics for machine learning practitioners, but it appears incremental as it builds on existing Choquet integral and weighting techniques without a paradigm shift.
The paper tackles the problem of capturing non-linear relationships and feature interactions in distance-based supervised learning by introducing feature subset weighting through the Choquet integral, resulting in a flexible distance metric that is computationally feasible and robust to duplicate or correlated features. It demonstrates improved performance in KNN classification compared to Mahalanobis and weighted distance methods, though specific numerical gains are not detailed.
This paper introduces feature subset weighting using monotone measures for distance-based supervised learning. The Choquet integral is used to define a distance metric that incorporates these weights. This integration enables the proposed distances to effectively capture non-linear relationships and account for interactions both between conditional and decision attributes and among conditional attributes themselves, resulting in a more flexible distance measure. In particular, we show how this approach ensures that the distances remain unaffected by the addition of duplicate and strongly correlated features. Another key point of this approach is that it makes feature subset weighting computationally feasible, since only $m$ feature subset weights should be calculated each time instead of calculating all feature subset weights ($2^m$), where $m$ is the number of attributes. Next, we also examine how the use of the Choquet integral for measuring similarity leads to a non-equivalent definition of distance. The relationship between distance and similarity is further explored through dual measures. Additionally, symmetric Choquet distances and similarities are proposed, preserving the classical symmetry between similarity and distance. Finally, we introduce a concrete feature subset weighting distance, evaluate its performance in a $k$-nearest neighbors (KNN) classification setting, and compare it against Mahalanobis distances and weighted distance methods.