Subspace Determination through Local Intrinsic Dimensional Decomposition: Theory and Experimentation
This addresses the challenge of efficiently finding relevant subspaces for clustering in high-dimensional data, though it appears incremental as it builds on existing LID models.
The paper tackles the problem of identifying feature subsets that contribute most to local neighborhoods in data, using a Local Intrinsic Dimension (LID) decomposition to indicate axis-aligned subspaces that support cluster formation.
Axis-aligned subspace clustering generally entails searching through enormous numbers of subspaces (feature combinations) and evaluation of cluster quality within each subspace. In this paper, we tackle the problem of identifying subsets of features with the most significant contribution to the formation of the local neighborhood surrounding a given data point. For each point, the recently-proposed Local Intrinsic Dimension (LID) model is used in identifying the axis directions along which features have the greatest local discriminability, or equivalently, the fewest number of components of LID that capture the local complexity of the data. In this paper, we develop an estimator of LID along axis projections, and provide preliminary evidence that this LID decomposition can indicate axis-aligned data subspaces that support the formation of clusters.