Risk Bounds for Robust Deep Learning
This work addresses the need for theoretical guarantees in robust deep learning, offering foundational insights that could benefit practitioners dealing with noisy or flawed data.
The paper tackles the problem of making deep learning robust to data flaws by providing statistical theory supporting the use of robust loss functions, showing that empirical-risk minimization with unbounded, Lipschitz-continuous loss functions can enable efficient prediction under minimal data assumptions.
It has been observed that certain loss functions can render deep-learning pipelines robust against flaws in the data. In this paper, we support these empirical findings with statistical theory. We especially show that empirical-risk minimization with unbounded, Lipschitz-continuous loss functions, such as the least-absolute deviation loss, Huber loss, Cauchy loss, and Tukey's biweight loss, can provide efficient prediction under minimal assumptions on the data. More generally speaking, our paper provides theoretical evidence for the benefits of robust loss functions in deep learning.