MNIST-Nd: a set of naturalistic datasets to benchmark clustering across dimensions
This provides a new benchmark for researchers in fields like biology to evaluate clustering performance across dimensions, though it is incremental as it builds on existing MNIST data and methods.
The paper tackles the problem of benchmarking clustering algorithms in high-dimensional data by introducing MNIST-Nd, a set of synthetic datasets with varying dimensions (2 to 64) that mimic real-world noise and imperfect cluster separation, and preliminary results show Leiden as the most robust algorithm for increasing dimensions.
Driven by advances in recording technology, large-scale high-dimensional datasets have emerged across many scientific disciplines. Especially in biology, clustering is often used to gain insights into the structure of such datasets, for instance to understand the organization of different cell types. However, clustering is known to scale poorly to high dimensions, even though the exact impact of dimensionality is unclear as current benchmark datasets are mostly two-dimensional. Here we propose MNIST-Nd, a set of synthetic datasets that share a key property of real-world datasets, namely that individual samples are noisy and clusters do not perfectly separate. MNIST-Nd is obtained by training mixture variational autoencoders with 2 to 64 latent dimensions on MNIST, resulting in six datasets with comparable structure but varying dimensionality. It thus offers the chance to disentangle the impact of dimensionality on clustering. Preliminary common clustering algorithm benchmarks on MNIST-Nd suggest that Leiden is the most robust for growing dimensions.