A unified framework for hard and soft clustering with regularized optimal transport
This provides a unified framework for clustering tasks, offering incremental improvements in performance and flexibility for machine learning practitioners.
The paper tackles the problem of inferring Finite Mixture Models from discrete data by formulating it as an optimal transport problem with entropic regularization, unifying hard and soft clustering and recovering the EM algorithm at λ=1. Experiments show that λ>1 improves inference performance and λ→0 enhances classification.
In this paper, we formulate the problem of inferring a Finite Mixture Model from discrete data as an optimal transport problem with entropic regularization of parameter $λ\geq 0$. Our method unifies hard and soft clustering, the Expectation-Maximization (EM) algorithm being exactly recovered for $λ=1$. The family of clustering algorithm we propose rely on the resolution of nonconvex problems using alternating minimization. We study the convergence property of our generalized $λ-$EM algorithms and show that each step in the minimization process has a closed form solution when inferring finite mixture models of exponential families. Experiments highlight the benefits of taking a parameter $λ>1$ to improve the inference performance and $λ\to 0$ for classification.