Pierre-Emmanuel Jabin

Leonid Berlyand, Robert Creese, Pierre-Emmanuel Jabin et al.

We consider a system of interacting particles with random initial conditions. Continuum approximations of the system, based on truncations of the BBGKY hierarchy, are described and simulated for various initial distributions and types of interaction. Specifically, we compare the Mean Field Approximation (MFA), the Kirkwood Superposition Approximation (KSA), and a recently developed truncation of the BBGKY hierarchy (the Truncation Approximation - TA). We show that KSA and TA perform more accurately than MFA in capturing approximate distributions (histograms) obtained from Monte Carlo simulations. Furthermore, TA is more numerically stable and less computationally expensive than KSA.

Leonid Berlyand, Robert Creese, Pierre-Emmanuel Jabin

We introduce two-scale loss functions for use in various gradient descent algorithms applied to classification problems via deep neural networks. This new method is generic in the sense that it can be applied to a wide range of machine learning architectures, from deep neural networks to support vector machines for example. These two-scale loss functions allow to focus the training onto objects in the training set which are not well classified. This leads to an increase in several measures of performance for appropriately-defined two-scale loss functions with respect to the more classical cross-entropy when tested on traditional deep neural networks on the MNIST, CIFAR10, and CIFAR100 data-sets.

Leonid Berlyand, Pierre-Emmanuel Jabin, C. Alex Safsten

We examine the stability of loss-minimizing training processes that are used for deep neural networks (DNN) and other classifiers. While a classifier is optimized during training through a so-called loss function, the performance of classifiers is usually evaluated by some measure of accuracy, such as the overall accuracy which quantifies the proportion of objects that are well classified. This leads to the guiding question of stability: does decreasing loss through training always result in increased accuracy? We formalize the notion of stability, and provide examples of instability. Our main result consists of two novel conditions on the classifier which, if either is satisfied, ensure stability of training, that is we derive tight bounds on accuracy as loss decreases. We also derive a sufficient condition for stability on the training set alone, identifying flat portions of the data manifold as potential sources of instability. The latter condition is explicitly verifiable on the training dataset. Our results do not depend on the algorithm used for training, as long as loss decreases with training.

Wojciech Czaja, Dong Dong, Pierre-Emmanuel Jabin et al.

We present a new feature extraction method for complex and large datasets, based on the concept of transport operators on graphs. The proposed approach generalizes and extends the many existing data representation methodologies built upon diffusion processes, to a new domain where dynamical systems play a key role. The main advantage of this approach comes from the ability to exploit different relationships than those arising in the context of e.g., Graph Laplacians. Fundamental properties of the transport operators are proved. We demonstrate the flexibility of the method by introducing several diverse examples of transformations. We close the paper with a series of computational experiments and applications to the problem of classification of hyperspectral satellite imagery, to illustrate the practical implications of our algorithm and its ability to quantify new aspects of relationships within complicated datasets.

Leonid Berlyand, Pierre-Emmanuel Jabin, Mykhailo Potomkin

We consider the motion of interacting particles governed by a coupled system of ODEs with random initial conditions. Direct computations for such systems are prohibitively expensive due to a very large number of particles and randomness requiring many realizations in their locations in the presence of strong interactions. While there are several approaches that address the above difficulties, none addresses all three simultaneously. Our goal is to develop such a computational approach in order to capture the experimentally observed emergence of correlations in the collective state (patterns due to strong interactions). Our approach is based on the truncation of the BBGKY hierarchy that allows one to go beyond the classical Mean Field limit and capture correlations while drastically reducing the computational complexity. Finally, we provide an example showing a numerical solution of this nonlinear and non-local system.