David Glickenstein

Stefan W. von Deylen, David Glickenstein, Max Wardetzky

We define barycentric coordinates on a Riemannian manifold using Karcher's center of mass technique applied to point masses for n+1 sufficiently close points, determining an n-dimensional Riemannian simplex defined as a "Karcher simplex." Specifically, a set of weights is mapped to the Riemannian center of mass for the corresponding point measures on the manifold with the given weights. If the points lie sufficiently close and in general position, this map is smooth and injective, giving a coordinate chart. We are then able to compute first and second derivative estimates of the coordinate chart. These estimates allow us to compare the Riemannian metric with the Euclidean metric induced on a simplex with edge lengths determined by the distances between the points. We show that these metrics differ by an error that shrinks quadratically with the maximum edge length. With such estimates, one can deduce convergence results for finite element approximations of problems on Riemannian manifolds.

Brian Bell, Michael Geyer, David Glickenstein et al.

We explore the equivalence between neural networks and kernel methods by deriving the first exact representation of any finite-size parametric classification model trained with gradient descent as a kernel machine. We compare our exact representation to the well-known Neural Tangent Kernel (NTK) and discuss approximation error relative to the NTK and other non-exact path kernel formulations. We experimentally demonstrate that the kernel can be computed for realistic networks up to machine precision. We use this exact kernel to show that our theoretical contribution can provide useful insights into the predictions made by neural networks, particularly the way in which they generalize.

Brian Bell, Michael Geyer, David Glickenstein et al.

There are a number of hypotheses underlying the existence of adversarial examples for classification problems. These include the high-dimensionality of the data, high codimension in the ambient space of the data manifolds of interest, and that the structure of machine learning models may encourage classifiers to develop decision boundaries close to data points. This article proposes a new framework for studying adversarial examples that does not depend directly on the distance to the decision boundary. Similarly to the smoothed classifier literature, we define a (natural or adversarial) data point to be $(γ,σ)$-stable if the probability of the same classification is at least $γ$ for points sampled in a Gaussian neighborhood of the point with a given standard deviation $σ$. We focus on studying the differences between persistence metrics along interpolants of natural and adversarial points. We show that adversarial examples have significantly lower persistence than natural examples for large neural networks in the context of the MNIST and ImageNet datasets. We connect this lack of persistence with decision boundary geometry by measuring angles of interpolants with respect to decision boundaries. Finally, we connect this approach with robustness by developing a manifold alignment gradient metric and demonstrating the increase in robustness that can be achieved when training with the addition of this metric.

Rebecca Faust, David Glickenstein, Carlos Scheidegger

Non-linear dimensionality reduction (NDR) methods such as LLE and t-SNE are popular with visualization researchers and experienced data analysts, but present serious problems of interpretation. In this paper, we present DimReader, a technique that recovers readable axes from such techniques. DimReader is based on analyzing infinitesimal perturbations of the dataset with respect to variables of interest. The perturbations define exactly how we want to change each point in the original dataset and we measure the effect that these changes have on the projection. The recovered axes are in direct analogy with the axis lines (grid lines) of traditional scatterplots. We also present methods for discovering perturbations on the input data that change the projection the most. The calculation of the perturbations is efficient and easily integrated into programs written in modern programming languages. We present results of DimReader on a variety of NDR methods and datasets both synthetic and real-life, and show how it can be used to compare different NDR methods. Finally, we discuss limitations of our proposal and situations where further research is needed.