Relative Deviation Learning Bounds and Generalization with Unbounded Loss Functions
This work provides theoretical foundations for machine learning tasks involving unbounded losses, which is incremental but useful for researchers in statistical learning theory.
The paper tackles the problem of deriving generalization bounds for learning algorithms with unbounded loss functions, presenting two-sided inequalities and proofs under the assumption of bounded moments, with applications in importance weighting and unbounded regression.
We present an extensive analysis of relative deviation bounds, including detailed proofs of two-sided inequalities and their implications. We also give detailed proofs of two-sided generalization bounds that hold in the general case of unbounded loss functions, under the assumption that a moment of the loss is bounded. These bounds are useful in the analysis of importance weighting and other learning tasks such as unbounded regression.