Conformal Prediction based Spectral Clustering
This work addresses the challenge of defining effective similarity measures in spectral clustering for data analysis, but it appears incremental as it builds on existing methods with a new affinity formulation.
The authors tackled the problem of improving spectral clustering by proposing a novel affinity measure based on non-conformity from Conformal Prediction, which captures neighborhood relationships and contextual similarity, and they showed it yields good results comparable to state-of-the-art methods.
Spectral Clustering(SC) is a prominent data clustering technique of recent times which has attracted much attention from researchers. It is a highly data-driven method and makes no strict assumptions on the structure of the data to be clustered. One of the central pieces of spectral clustering is the construction of an affinity matrix based on a similarity measure between data points. The way the similarity measure is defined between data points has a direct impact on the performance of the SC technique. Several attempts have been made in the direction of strengthening the pairwise similarity measure to enhance the spectral clustering. In this work, we have defined a novel affinity measure by employing the concept of non-conformity used in Conformal Prediction(CP) framework. The non-conformity based affinity captures the relationship between neighborhoods of data points and has the power to generalize the notion of contextual similarity. We have shown that this formulation of affinity measure gives good results and compares well with the state of the art methods.