Semidefinite programming relaxations and debiasing for MAXCUT-based clustering
This work addresses clustering in high-dimensional, low-signal settings for data science applications, offering incremental improvements in theoretical guarantees.
The paper tackles the problem of partitioning data from a mixture of two sub-gaussian distributions using semidefinite programming relaxations of a MAXCUT-based clustering method, proving that the misclassification error decays exponentially with the signal-to-noise ratio for one algorithm and introducing a debiased estimator that removes a covariance assumption.
In this paper, we consider the problem of partitioning a small data sample of size $n$ drawn from a mixture of 2 sub-gaussian distributions in $\R^p$. We consider semidefinite programming relaxations of an integer quadratic program that is formulated as finding the maximum cut on a graph, where edge weights in the cut represent dissimilarity scores between two nodes based on their $p$ features. We are interested in the case that individual features are of low average quality $γ$, and we want to use as few of them as possible to correctly partition the sample. Denote by $Δ^2:=p γ$ the $\ell_2^2$ distance between two centers (mean vectors) in $\R^p$. The goal is to allow a full range of tradeoffs between $n, p, γ$ in the sense that partial recovery (success rate $< 100%$) is feasible once the signal to noise ratio $s^2 := \min{np γ^2, Δ^2}$ is lower bounded by a constant. For both balanced and unbalanced cases, we allow each population to have distinct covariance structures with diagonal matrices as special cases. In the present work, (a) we provide a unified framework for analyzing three computationally efficient algorithms, namely, SDP1, BalancedSDP, and Spectral clustering; and (b) we prove that the misclassification error decays exponentially with respect to the SNR $s^2$ for SDP1. Moreover, for balanced partitions, we design an estimator $\bf {BalancedSDP}$ with a superb debiasing property. Indeed, with this new estimator, we remove an assumption (A2) on bounding the trace difference between the two population covariance matrices while proving the exponential error bound as stated above. These estimators and their statistical analyses are novel to the best of our knowledge. We provide simulation evidence illuminating the theoretical predictions.