Improved PAC-Bayesian Bounds for Linear Regression
This work provides incremental theoretical improvements for researchers in statistical learning theory, specifically enhancing bounds for linear regression under more realistic data assumptions.
The paper tackles the problem of deriving tighter PAC-Bayesian error bounds for linear regression, achieving improvements that allow convergence to generalization loss and applicability to non-independent data like time series from ARX models.
In this paper, we improve the PAC-Bayesian error bound for linear regression derived in Germain et al. [10]. The improvements are twofold. First, the proposed error bound is tighter, and converges to the generalization loss with a well-chosen temperature parameter. Second, the error bound also holds for training data that are not independently sampled. In particular, the error bound applies to certain time series generated by well-known classes of dynamical models, such as ARX models.