Bayesian linear regression with Student-t assumptions
This work provides an incremental improvement for researchers and practitioners in machine learning by offering a more flexible Bayesian regression model with faster convergence in specific tasks.
The paper tackles the problem of Bayesian linear regression with non-Gaussian assumptions by proposing a model with Student-t assumptions (BLRS) that allows exact inference, generalizing conjugate priors and the EM algorithm, and shows that its q-EM algorithm converges faster than the standard Gaussian-based EM for predicting online news popularity.
As an automatic method of determining model complexity using the training data alone, Bayesian linear regression provides us a principled way to select hyperparameters. But one often needs approximation inference if distribution assumption is beyond Gaussian distribution. In this paper, we propose a Bayesian linear regression model with Student-t assumptions (BLRS), which can be inferred exactly. In this framework, both conjugate prior and expectation maximization (EM) algorithm are generalized. Meanwhile, we prove that the maximum likelihood solution is equivalent to the standard Bayesian linear regression with Gaussian assumptions (BLRG). The $q$-EM algorithm for BLRS is nearly identical to the EM algorithm for BLRG. It is showed that $q$-EM for BLRS can converge faster than EM for BLRG for the task of predicting online news popularity.