End-to-end Differentiable Clustering with Associative Memories
This addresses the challenge of making clustering differentiable for integration with deep learning, offering a novel method with significant performance gains, though it appears incremental in combining existing AMs with clustering.
The paper tackled the problem of clustering as an intensive discrete optimization by proposing a novel continuous relaxation using Associative Memories, enabling end-to-end differentiable clustering called ClAM, which improved upon traditional k-means and recent relaxations by up to 60% in Silhouette Coefficient.
Clustering is a widely used unsupervised learning technique involving an intensive discrete optimization problem. Associative Memory models or AMs are differentiable neural networks defining a recursive dynamical system, which have been integrated with various deep learning architectures. We uncover a novel connection between the AM dynamics and the inherent discrete assignment necessary in clustering to propose a novel unconstrained continuous relaxation of the discrete clustering problem, enabling end-to-end differentiable clustering with AM, dubbed ClAM. Leveraging the pattern completion ability of AMs, we further develop a novel self-supervised clustering loss. Our evaluations on varied datasets demonstrate that ClAM benefits from the self-supervision, and significantly improves upon both the traditional Lloyd's k-means algorithm, and more recent continuous clustering relaxations (by upto 60% in terms of the Silhouette Coefficient).