Classification of Data Generated by Gaussian Mixture Models Using Deep ReLU Networks
It provides theoretical verification for the efficiency of deep neural networks in practical classification problems, such as speech, image, and text data, but is incremental as it builds on existing generalization analysis by extending to unbounded domains.
This paper tackles the binary classification of unbounded data from Gaussian Mixture Models using deep ReLU networks, deriving non-asymptotic upper bounds and convergence rates for excess risk that are independent of dimension, demonstrating the ability to overcome the curse of dimensionality.
This paper studies the binary classification of unbounded data from ${\mathbb R}^d$ generated under Gaussian Mixture Models (GMMs) using deep ReLU neural networks. We obtain $\unicode{x2013}$ for the first time $\unicode{x2013}$ non-asymptotic upper bounds and convergence rates of the excess risk (excess misclassification error) for the classification without restrictions on model parameters. The convergence rates we derive do not depend on dimension $d$, demonstrating that deep ReLU networks can overcome the curse of dimensionality in classification. While the majority of existing generalization analysis of classification algorithms relies on a bounded domain, we consider an unbounded domain by leveraging the analyticity and fast decay of Gaussian distributions. To facilitate our analysis, we give a novel approximation error bound for general analytic functions using ReLU networks, which may be of independent interest. Gaussian distributions can be adopted nicely to model data arising in applications, e.g., speeches, images, and texts; our results provide a theoretical verification of the observed efficiency of deep neural networks in practical classification problems.