Learning Probabilistic Programs Using Backpropagation
This addresses the problem of improving probabilistic modeling performance for researchers and practitioners in machine learning, though it appears incremental as it adapts existing backpropagation techniques to a new context.
The paper tackles the challenge of learning probabilistic models, which are difficult to train and underperform compared to deep neural networks, by introducing a method that uses backpropagation to learn probabilistic program parameters, enabling deep probabilistic programming models to be trained similarly to neural networks.
Probabilistic modeling enables combining domain knowledge with learning from data, thereby supporting learning from fewer training instances than purely data-driven methods. However, learning probabilistic models is difficult and has not achieved the level of performance of methods such as deep neural networks on many tasks. In this paper, we attempt to address this issue by presenting a method for learning the parameters of a probabilistic program using backpropagation. Our approach opens the possibility to building deep probabilistic programming models that are trained in a similar way to neural networks.