Convergence Rates for Gaussian Mixtures of Experts
This work provides foundational theoretical insights for statistical learning in mixture models, addressing a specific bottleneck in understanding estimation rates.
The paper tackles the problem of establishing convergence rates for maximum likelihood estimation in over-specified Gaussian mixtures of experts with covariate-free gating networks, deriving theoretical bounds and minimax lower bounds for parameter estimation.
We provide a theoretical treatment of over-specified Gaussian mixtures of experts with covariate-free gating networks. We establish the convergence rates of the maximum likelihood estimation (MLE) for these models. Our proof technique is based on a novel notion of \emph{algebraic independence} of the expert functions. Drawing on optimal transport theory, we establish a connection between the algebraic independence and a certain class of partial differential equations (PDEs). Exploiting this connection allows us to derive convergence rates and minimax lower bounds for parameter estimation.