Distribution Learnability and Robustness
This work addresses foundational issues in machine learning theory for researchers, revealing limitations in distribution learning robustness.
The paper tackles the problem of distribution learnability by showing that realizable learnability does not imply agnostic learnability, unlike in other settings like PAC learning, and demonstrates that learnability is robust only to additive corruption, not subtractive corruption.
We examine the relationship between learnability and robust (or agnostic) learnability for the problem of distribution learning. We show that, contrary to other learning settings (e.g., PAC learning of function classes), realizable learnability of a class of probability distributions does not imply its agnostic learnability. We go on to examine what type of data corruption can disrupt the learnability of a distribution class and what is such learnability robust against. We show that realizable learnability of a class of distributions implies its robust learnability with respect to only additive corruption, but not against subtractive corruption. We also explore related implications in the context of compression schemes and differentially private learnability.