Thomas Trogdon

Percy Deift, Govind Menon, Sheehan Olver et al.

The authors present evidence for universality in numerical computations with random data. Given a (possibly stochastic) numerical algorithm with random input data, the time (or number of iterations) to convergence (within a given tolerance) is a random variable, called the halting time. Two-component universality is observed for the fluctuations of the halting time, i.e., the histogram for the halting times, centered by the sample average and scaled by the sample variance, collapses to a universal curve, independent of the input data distribution, as the dimension increases. Thus, up to two components, the sample average and the sample variance, the statistics for the halting time are universally prescribed. The case studies include six standard numerical algorithms, as well as a model of neural computation and decision making. A link to relevant software is provided in for the reader who would like to do computations of his'r own.

Percy Deift, Thomas Trogdon

We prove universality for the fluctuations of the halting time for the Toda algorithm to compute the largest eigenvalue of real symmetric and complex Hermitian matrices. The proof relies on recent results on the statistics of the eigenvalues and eigenvectors of random matrices (such as delocalization, rigidity and edge universality) in a crucial way.

Percy Deift, Thomas Trogdon

We prove a universal limit theorem for the halting time, or iteration count, of the power/inverse power methods and the QR eigenvalue algorithm. Specifically, we analyze the required number of iterations to compute extreme eigenvalues of random, positive-definite sample covariance matrices to within a prescribed tolerance. The universality theorem provides a complexity estimate for the algorithms which, in this random setting, holds with high probability. The method of proof relies on recent results on the statistics of the eigenvalues and eigenvectors of random sample covariance matrices (i.e., delocalization, rigidity and edge universality).

Sheehan Olver, Thomas Trogdon

The effective and efficient numerical solution of Riemann-Hilbert problems has been demonstrated in recent work. With the aid of ideas from the method of nonlinear steepest descent for Riemann-Hilbert problems, the resulting numerical methods have been shown numerically to retain accuracy as values of certain parameters become arbitrarily large. Remarkably, this numerical approach does not require knowledge of local parametrices; rather, the deformed contour is scaled near stationary points at a specific rate. The primary aim of this paper is to prove that this observed asymptotic accuracy is indeed achieved. To do so, we first construct a general theoretical framework for the numerical solution of Riemann-Hilbert problems. Second, we demonstrate the precise link between nonlinear steepest descent and the success of numerics in asymptotic regimes. In particular, we prove sufficient conditions for numerical methods to retain accuracy. Finally, we compute solutions to the homogeneous Painlevé II equation and the modified Korteweg-de Vries equations to explicitly demonstrate the practical validity of the theory.

Percy Deift, Thomas Trogdon

We discuss various universality aspects of numerical computations using standard algorithms. These aspects include empirical observations and rigorous results. We also make various speculations about computation in a broader sense.

Sheehan Olver, Thomas Trogdon

In recent developments, a general approach for solving Riemann--Hilbert problems numerically has been developed. We review this numerical framework, and apply it to the calculation of orthogonal polynomials on the real line. Combining this numerical algorithm with an approach to compute Fredholm determinants, we are able to calculate level densities and gap statistics for general finite-dimensional unitary ensembles. We also include a description of how to compute the Hastings--McLeod solution of the homogeneous Painlevé II equation.

Deniz Bilman, Thomas Trogdon

We present a method to compute the inverse scattering transform (IST) for the famed Toda lattice by solving the associated Riemann--Hilbert (RH) problem numerically. Deformations for the RH problem are incorporated so that the IST can be evaluated in $\mathcal O(1)$ operations for arbitrary points in the $(n,t)$-domain, including short- and long-time regimes. No time-stepping is required to compute the solution because $(n,t)$ appear as parameters in the associated RH problem. The solution of the Toda lattice is computed in long-time asymptotic regions where the asymptotics are not known rigorously.

Govind Menon, Thomas Trogdon

The purpose of this paper is to establish bounds on the rate of convergence of the conjugate gradient algorithm when the underlying matrix is a random positive definite perturbation of a deterministic positive definite matrix. We estimate all finite moments of a natural halting time when the random perturbation is drawn from the Laguerre unitary ensemble in a critical scaling regime explored in Deift et al. (2016). These estimates are used to analyze the expected iteration count in the framework of smoothed analysis, introduced by Spielman and Teng (2001). The rigorous results are compared with numerical calculations in several cases of interest.

Deniz Bilman, Thomas Trogdon

We compare performances of well-known numerical time-stepping methods that are widely used to compute solutions of the doubly-infinite Fermi-Pasta-Ulam-Tsingou (FPUT) lattice equations. The methods are benchmarked according to (1) their accuracy in capturing the soliton peaks and (2) in capturing highly-oscillatory parts of the solutions of the Toda lattice resulting from a variety of initial data. The numerical inverse scattering transform method is used to compute a reference solution with high accuracy. We find that benchmarking a numerical method on pure-soliton initial data can lead one to overestimate the accuracy of the method.

Percy Deift, Stephen D. Miller, Thomas Trogdon

We consider the normalized distribution of the overall running times of some cryptographic algorithms, and what information they reveal about the algorithms. Recent work of Deift, Menon, Olver, Pfrang, and Trogdon has shown that certain numerical algorithms applied to large random matrices exhibit a characteristic distribution of running times, which depends only on the algorithm but are independent of the choice of probability distributions for the matrices. Different algorithms often exhibit different running time distributions, and so the histograms for these running time distributions provide a time-signature for the algorithms, making it possible, in many cases, to distinguish one algorithm from another. In this paper we extend this analysis to cryptographic algorithms, and present examples of such algorithms with time-signatures that are indistinguishable, and others with time-signatures that are clearly distinct.

Levent Sagun, Thomas Trogdon, Yann LeCun

The authors present empirical distributions for the halting time (measured by the number of iterations to reach a given accuracy) of optimization algorithms applied to two random systems: spin glasses and deep learning. Given an algorithm, which we take to be both the optimization routine and the form of the random landscape, the fluctuations of the halting time follow a distribution that, after centering and scaling, remains unchanged even when the distribution on the landscape is changed. We observe two qualitative classes: A Gumbel-like distribution that appears in Google searches, human decision times, the QR eigenvalue algorithm and spin glasses, and a Gaussian-like distribution that appears in conjugate gradient method, deep network with MNIST input data and deep network with random input data. This empirical evidence suggests presence of a class of distributions for which the halting time is independent of the underlying distribution under some conditions.

Alex Townsend, Thomas Trogdon, Sheehan Olver

A fast and accurate algorithm for the computation of Gauss-Hermite and generalized Gauss-Hermite quadrature nodes and weights is presented. The algorithm is based on Newton's method with carefully selected initial guesses for the nodes and a fast evaluation scheme for the associated orthogonal polynomial. In the Gauss-Hermite case the initial guesses and evaluation scheme rely on explicit asymptotic formulas. For generalized Gauss-Hermite, the initial guesses are furnished by sampling a certain equilibrium measure and the associated polynomial evaluated via a Riemann-Hilbert reformulation. In both cases the $n$-point quadrature rule is computed in $\mathcal{O}(n)$ operations to an accuracy that is close to machine precision. For sufficiently large $n$, some of the quadrature weights have a value less than the smallest positive normalized floating-point number in double precision and we exploit this fact to achieve a complexity as low as $\mathcal{O}(\sqrt{n})$.