A Note on the Efficient Evaluation of PAC-Bayes Bounds
This work addresses a computational bottleneck in PAC-Bayes theory for researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the high computational cost of evaluating PAC-Bayes bounds by proposing a general alternative method that reduces computational requirements by an order of the dataset size, achieving significant efficiency gains.
When utilising PAC-Bayes theory for risk certification, it is usually necessary to estimate and bound the Gibbs risk of the PAC-Bayes posterior. Many works in the literature employ a method for this which requires a large number of passes of the dataset, incurring high computational cost. This manuscript presents a very general alternative which makes computational savings on the order of the dataset size.