Sina Baghal

Sina Baghal

We present a non-asymptotic concentration inequality for the random matrix product \begin{equation}\label{eq:Zn} Z_n = \left(I_d-αX_n\right)\left(I_d-αX_{n-1}\right)\cdots \left(I_d-αX_1\right), \end{equation} where $\left\{X_k \right\}_{k=1}^{+\infty}$ is a sequence of bounded independent random positive semidefinite matrices with common expectation $\mathbb{E}\left[X_k\right]=Σ$. Under these assumptions, we show that, for small enough positive $α$, $Z_n$ satisfies the concentration inequality \begin{equation}\label{eq:CTbound} \mathbb{P}\left(\left\Vert Z_n-\mathbb{E}\left[Z_n\right]\right\Vert \geq t\right) \leq 2d^2\cdot\exp\left(\frac{-t^2}{ασ^2} \right) \quad \text{for all } t\geq 0, \end{equation} where $σ^2$ denotes a variance parameter.

Sina Baghal, Courtney Paquette, Stephen A. Vavasis

We propose a new, simple, and computationally inexpensive termination test for constant step-size stochastic gradient descent (SGD) applied to binary classification on the logistic and hinge loss with homogeneous linear predictors. Our theoretical results support the effectiveness of our stopping criterion when the data is Gaussian distributed. This presence of noise allows for the possibility of non-separable data. We show that our test terminates in a finite number of iterations and when the noise in the data is not too large, the expected classifier at termination nearly minimizes the probability of misclassification. Finally, numerical experiments indicate for both real and synthetic data sets that our termination test exhibits a good degree of predictability on accuracy and running time.