Hao Ji

Hao Ji, Wenjian Yu, Yaohang Li

In this paper, we present a Rank Revealing Randomized Singular Value Decomposition (R3SVD) algorithm to incrementally construct a low-rank approximation of a potentially large matrix while adaptively estimating the appropriate rank that can capture most of the actions of the matrix. Starting from a low-rank approximation with an initial guessed rank, R3SVD adopts an orthogonal Gaussian sampling approach to obtain the dominant subspace within the leftover space, which is used to add up to the existing low-rank approximation. Orthogonal Gaussian sampling is repeated until an appropriate low-rank approximation with satisfactory accuracy, measured by the overall energy percentage of the original matrix, is obtained. While being a fast algorithm, R3SVD is also a memory-aware algorithm where the computational process can be decomposed into a series of sampling tasks that use constant amount of memory. Numerical examples in image compression and matrix completion are used to demonstrate the effectiveness of R3SVD in low-rank approximation.

Hao Ji, Seth H. Weinberg, Yaohang Li

In this paper, we revisit the generalized block power methods for approximating the eigenvector associated with $λ_1 = 1$ of a Markov chain transition matrix. Our analysis of the block power method shows that when $s$ linearly independent probability vectors are used as the initial block, the convergence of the block power method to the stationary distribution depends on the magnitude of the $(s+1)$th dominant eigenvalue $λ_{s+1}$ of $P$ instead of that of $λ_2$ in the power method. Therefore, the block power method with block size $s$ is particularly effective for transition matrices where $|λ_{s+1}|$ is well separated from $λ_1 = 1$ but $|λ_2|$ is not. This approach is particularly useful when visiting the elements of a large transition matrix is the main computational bottleneck over matrix--vector multiplications, where the block power method can effectively reduce the total number of times to pass over the matrix. To further reduce the overall computational cost, we combine the block power method with a sliding window scheme, taking advantage of the subsequent vectors of the latest $s$ iterations to assemble the block matrix. The sliding window scheme correlates vectors in the sliding window to quickly remove the influences from the eigenvalues whose magnitudes are smaller than $|λ_{s}|$ to reduce the overall number of matrix--vector multiplications to reach convergence. Finally, we compare the effectiveness of these methods in a Markov chain model representing a stochastic luminal calcium release site.

Eitan Rothberg, Tingting Chen, Luo Jie et al.

Today's state-of-the-art image classifiers fail to correctly classify carefully manipulated adversarial images. In this work, we develop a new, localized adversarial attack that generates adversarial examples by imperceptibly altering the backgrounds of normal images. We first use this attack to highlight the unnecessary sensitivity of neural networks to changes in the background of an image, then use it as part of a new training technique: localized adversarial training. By including locally adversarial images in the training set, we are able to create a classifier that suffers less loss than a non-adversarially trained counterpart model on both natural and adversarial inputs. The evaluation of our localized adversarial training algorithm on MNIST and CIFAR-10 datasets shows decreased accuracy loss on natural images, and increased robustness against adversarial inputs.