Distribution free M-estimation
This work addresses a foundational problem in statistics and machine learning by delineating assumption-free solvability, which is significant for researchers and practitioners seeking robust methods without distributional constraints.
The paper characterizes when convex M-estimation or stochastic optimization problems are solvable without assumptions on the data distribution, identifying precise conditions that distinguish solvable from unsolvable cases, and shows that Lipschitz continuity of the loss is not necessary for distribution-free minimization.
The basic question of delineating those statistical problems that are solvable without making any assumptions on the underlying data distribution has long animated statistics and learning theory. This paper characterizes when a convex M-estimation or stochastic optimization problem is solvable in such an assumption-free setting, providing a precise dividing line between solvable and unsolvable problems. The conditions we identify show, perhaps surprisingly, that Lipschitz continuity of the loss being minimized is not necessary for distribution free minimization, and they are also distinct from classical characterizations of learnability in machine learning.