How To Train Your Program: a Probabilistic Programming Pattern for Bayesian Learning From Data
This work addresses computational efficiency in Bayesian machine learning for practitioners using probabilistic programming, though it appears incremental as it builds on existing hierarchical modeling approaches.
The authors tackled the problem of Bayesian learning from data by introducing a probabilistic programming pattern called 'stump and fungus' that implements training as inference on a hierarchical model, achieving lower computational costs for obtaining posterior distributions on new data compared to traditional hierarchical modeling.
We present a Bayesian approach to machine learning with probabilistic programs. In our approach, training on available data is implemented as inference on a hierarchical model. The posterior distribution of model parameters is then used to \textit{stochastically condition} a complementary model, such that inference on new data yields the same posterior distribution of latent parameters corresponding to the new data as inference on a hierachical model on the combination of both previously available and new data, at a lower computation cost. We frame the approach as a design pattern of probabilistic programming referred to herein as `stump and fungus', and evaluate realization of the pattern on case studies.