Attention-Based Clustering: Learning a Kernel from Context
This addresses clustering tasks in machine learning, but it appears incremental as it builds on existing attention mechanisms and kernel-based methods.
The paper tackled the problem of clustering by proposing Attention-Based Clustering (ABC), a neural architecture that learns latent representations adapting to context within input sets, and it presented competitive results for clustering Omniglot characters.
In machine learning, no data point stands alone. We believe that context is an underappreciated concept in many machine learning methods. We propose Attention-Based Clustering (ABC), a neural architecture based on the attention mechanism, which is designed to learn latent representations that adapt to context within an input set, and which is inherently agnostic to input sizes and number of clusters. By learning a similarity kernel, our method directly combines with any out-of-the-box kernel-based clustering approach. We present competitive results for clustering Omniglot characters and include analytical evidence of the effectiveness of an attention-based approach for clustering.