Generalization of Hamiltonian algorithms
This provides theoretical guarantees for generalization in machine learning, particularly for stochastic algorithms, but is incremental as it extends existing frameworks.
The paper tackles the problem of proving generalization bounds for stochastic learning algorithms, establishing that such bounds hold when the algorithm's output distribution is absolutely continuous with a subgaussian Radon-Nikodym derivative, with applications including the Gibbs algorithm and PAC-Bayesian bounds.
The paper proves generalization results for a class of stochastic learning algorithms. The method applies whenever the algorithm generates an absolutely continuous distribution relative to some a-priori measure and the Radon Nikodym derivative has subgaussian concentration. Applications are bounds for the Gibbs algorithm and randomizations of stable deterministic algorithms as well as PAC-Bayesian bounds with data-dependent priors.