Robust Estimation under the Wasserstein Distance
This addresses the problem of robust statistical estimation for machine learning practitioners dealing with contaminated datasets, offering a theoretically grounded and scalable solution.
The paper tackles robust distribution estimation from adversarially corrupted samples under the Wasserstein distance by combining partial optimal transport and minimum distance estimation, achieving minimax-optimal risk and enabling robust generative modeling with WGANs through a novel penalized dual formulation.
We study the problem of robust distribution estimation under the Wasserstein distance, a popular discrepancy measure between probability distributions rooted in optimal transport (OT) theory. Given $n$ samples from an unknown distribution $μ$, of which $\varepsilon n$ are adversarially corrupted, we seek an estimate for $μ$ with minimal Wasserstein error. To address this task, we draw upon two frameworks from OT and robust statistics: partial OT (POT) and minimum distance estimation (MDE). We prove new structural properties for POT and use them to show that MDE under a partial Wasserstein distance achieves the minimax-optimal robust estimation risk in many settings. Along the way, we derive a novel dual form for POT that adds a sup-norm penalty to the classic Kantorovich dual for standard OT. Since the popular Wasserstein generative adversarial network (WGAN) framework implements Wasserstein MDE via Kantorovich duality, our penalized dual enables large-scale generative modeling with contaminated datasets via an elementary modification to WGAN. Numerical experiments demonstrating the efficacy of our approach in mitigating the impact of adversarial corruptions are provided.