Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching
This provides a more efficient method for prior specification in Bayesian matrix factorization, addressing a known bottleneck for researchers and practitioners in machine learning, though it is incremental as it builds on existing prior predictive concepts.
The paper tackles the problem of selecting prior distributions in Bayesian models without costly posterior inference by matching virtual statistics from the prior predictive distribution to user-provided targets, and demonstrates its application to probabilistic matrix factorization, including analytical solutions for Poisson factorization and empirical validation of sensitivity.
The behavior of many Bayesian models used in machine learning critically depends on the choice of prior distributions, controlled by some hyperparameters that are typically selected by Bayesian optimization or cross-validation. This requires repeated, costly, posterior inference. We provide an alternative for selecting good priors without carrying out posterior inference, building on the prior predictive distribution that marginalizes out the model parameters. We estimate virtual statistics for data generated by the prior predictive distribution and then optimize over the hyperparameters to learn ones for which these virtual statistics match target values provided by the user or estimated from (subset of) the observed data. We apply the principle for probabilistic matrix factorization, for which good solutions for prior selection have been missing. We show that for Poisson factorization models we can analytically determine the hyperparameters, including the number of factors, that best replicate the target statistics, and we study empirically the sensitivity of the approach for model mismatch. We also present a model-independent procedure that determines the hyperparameters for general models by stochastic optimization, and demonstrate this extension in context of hierarchical matrix factorization models.