Matt Jacobs

Matt Jacobs, Flavien Léger, Wuchen Li et al.

We present a primal-dual method to solve L1-type non-smooth optimization problems independently of the grid size. We apply these results to two important problems : the Rudin-Osher-Fatemi image denoising model and the L1 earth mover's distance from optimal transport. Crucially, we provide analysis that determines the choice of optimal step sizes and we prove that our method converges independently of the grid size. Our approach allows us to solve these problems on grids as large as 4096 by 4096 in a few minutes without parallelization.

Nicolas Garcia Trillos, Matt Jacobs, Jakwang Kim

We study a family of adversarial multiclass classification problems and provide equivalent reformulations in terms of: 1) a family of generalized barycenter problems introduced in the paper and 2) a family of multimarginal optimal transport problems where the number of marginals is equal to the number of classes in the original classification problem. These new theoretical results reveal a rich geometric structure of adversarial learning problems in multiclass classification and extend recent results restricted to the binary classification setting. A direct computational implication of our results is that by solving either the barycenter problem and its dual, or the MOT problem and its dual, we can recover the optimal robust classification rule and the optimal adversarial strategy for the original adversarial problem. Examples with synthetic and real data illustrate our results.

Nicolas Garcia Trillos, Matt Jacobs, Jakwang Kim

We study three models of the problem of adversarial training in multiclass classification designed to construct robust classifiers against adversarial perturbations of data in the agnostic-classifier setting. We prove the existence of Borel measurable robust classifiers in each model and provide a unified perspective of the adversarial training problem, expanding the connections with optimal transport initiated by the authors in previous work and developing new connections between adversarial training in the multiclass setting and total variation regularization. As a corollary of our results, we prove the existence of Borel measurable solutions to the agnostic adversarial training problem in the binary classification setting, a result that improves results in the literature of adversarial training, where robust classifiers were only known to exist within the enlarged universal $σ$-algebra of the feature space.

Emmanuel Gil Torres, Matt Jacobs, Xiangxiong Zhang

We prove an explicit local linear rate for ADMM solving the isotropic Total Variation (TV) norm compressed sensing problem in multiple dimensions, by analyzing the auxiliary variable in the equivalent Douglas-Rachford splitting on a dual problem. Numerical verification on large 3D problems and real MRI data will be shown. Though the proven rate is not sharp, it is close to the observed ones in numerical tests. The proven rate is not sharp, but it provides an explicit upper bound that appears close to the observed convergence rate in numerical experiments, although we do not claim this behavior holds in general.

Nicolas Garcia Trillos, Matt Jacobs, Jakwang Kim et al.

Despite the success of deep learning-based algorithms, it is widely known that neural networks may fail to be robust. A popular paradigm to enforce robustness is adversarial training (AT), however, this introduces many computational and theoretical difficulties. Recent works have developed a connection between AT in the multiclass classification setting and multimarginal optimal transport (MOT), unlocking a new set of tools to study this problem. In this paper, we leverage the MOT connection to propose computationally tractable numerical algorithms for computing universal lower bounds on the optimal adversarial risk and identifying optimal classifiers. We propose two main algorithms based on linear programming (LP) and entropic regularization (Sinkhorn). Our key insight is that one can harmlessly truncate the higher order interactions between classes, preventing the combinatorial run times typically encountered in MOT problems. We validate these results with experiments on MNIST and CIFAR-$10$, which demonstrate the tractability of our approach.

Leon Bungert, Nicolás García Trillos, Matt Jacobs et al.

Although deep neural networks have achieved super-human performance on many classification tasks, they often exhibit a worrying lack of robustness towards adversarially generated examples. Thus, considerable effort has been invested into reformulating standard Risk Minimization (RM) into an adversarially robust framework. Recently, attention has shifted towards approaches which interpolate between the robustness offered by adversarial training and the higher clean accuracy and faster training times of RM. In this paper, we take a fresh and geometric view on one such method -- Probabilistically Robust Learning (PRL). We propose a mathematical framework for understanding PRL, which allows us to identify geometric pathologies in its original formulation and to introduce a family of probabilistic nonlocal perimeter functionals to rectify them. We prove existence of solutions to the original and modified problems using novel relaxation methods and also study properties, as well as local limits, of the introduced perimeters. We also clarify, through a suitable $Γ$-convergence analysis, the way in which the original and modified PRL models interpolate between risk minimization and adversarial training.