Course Project Report: Comparing MCMC and Variational Inference for Bayesian Probabilistic Matrix Factorization on the MovieLens Dataset
This is an incremental course project comparing standard Bayesian inference methods for a specific domain (recommendation systems).
The authors tackled the problem of approximating the intractable posterior in Bayesian Probabilistic Matrix Factorization for recommendation systems by comparing Markov Chain Monte Carlo (MCMC) and Variational Inference (VI) on the MovieLens dataset, finding that VI converges faster but MCMC provides more accurate posterior estimates.
This is a course project report with complete methodology, experiments, references and mathematical derivations. Matrix factorization [1] is a widely used technique in recommendation systems. Probabilistic Matrix Factorization (PMF) [2] extends traditional matrix factorization by incorporating probability distributions over latent factors, allowing for uncertainty quantification. However, computing the posterior distribution is intractable due to the high-dimensional integral. To address this, we employ two Bayesian inference methods: Markov Chain Monte Carlo (MCMC) [3, 4] and Variational Inference (VI) [5, 6] to approximate the posterior. We evaluate their performance on MovieLens dataset [7] and compare their convergence speed, predictive accuracy, and computational efficiency. Experimental results demonstrate that VI offers faster convergence, while MCMC provides more accurate posterior estimates.