Metrics for quantifying isotropy in high dimensional unsupervised clustering tasks in a materials context
This work addresses the challenge of evaluating clustering quality in high-dimensional data for materials science and broader ML applications, though it is incremental as it builds on existing isotropy measures.
The paper tackles the problem of quantifying cluster isotropy in high-dimensional unsupervised clustering, particularly for materials science, by analyzing and expanding measures of isotropy and demonstrating their application to the Inorganic Crystal Structure Database and MNIST dataset, with Python implementations provided.
Clustering is a common task in machine learning, but clusters of unlabelled data can be hard to quantify. The application of clustering algorithms in chemistry is often dependant on material representation. Ascertaining the effects of different representations, clustering algorithms, or data transformations on the resulting clusters is difficult due to the dimensionality of these data. We present a thorough analysis of measures for isotropy of a cluster, including a novel implantation based on an existing derivation. Using fractional anisotropy, a common method used in medical imaging for comparison, we then expand these measures to examine the average isotropy of a set of clusters. A use case for such measures is demonstrated by quantifying the effects of kernel approximation functions on different representations of the Inorganic Crystal Structure Database. Broader applicability of these methods is demonstrated in analysing learnt embedding of the MNIST dataset. Random clusters are explored to examine the differences between isotropy measures presented, and to see how each method scales with the dimensionality. Python implementations of these measures are provided for use by the community.