Overspecified Mixture Discriminant Analysis: Exponential Convergence, Statistical Guarantees, and Remote Sensing Applications
It provides a theoretical framework for overspecified MDA, which is empirically used in complex data like remote sensing, but the work is incremental as it extends existing analysis to specific conditions.
This study analyzes the classification error of Mixture Discriminant Analysis (MDA) when using more mixture components than needed, showing that with proper initialization, the Expectation-Maximization algorithm converges exponentially fast to the Bayes risk at the population level and achieves a rate of n^{-1/2} for finite samples.
This study explores the classification error of Mixture Discriminant Analysis (MDA) in scenarios where the number of mixture components exceeds those present in the actual data distribution, a condition known as overspecification. We use a two-component Gaussian mixture model within each class to fit data generated from a single Gaussian, analyzing both the algorithmic convergence of the Expectation-Maximization (EM) algorithm and the statistical classification error. We demonstrate that, with suitable initialization, the EM algorithm converges exponentially fast to the Bayes risk at the population level. Further, we extend our results to finite samples, showing that the classification error converges to Bayes risk with a rate $n^{-1/2}$ under mild conditions on the initial parameter estimates and sample size. This work provides a rigorous theoretical framework for understanding the performance of overspecified MDA, which is often used empirically in complex data settings, such as image and text classification. To validate our theory, we conduct experiments on remote sensing datasets.