Amortized Inference of Variational Bounds for Learning Noisy-OR
This work addresses inference efficiency for probabilistic modeling, but it is incremental as it builds on established methods.
The paper tackles the problem of approximate inference in probabilistic models by proposing Amortized Conjugate Posterior (ACP), a hybrid method that combines classical variational distributions with amortized inference, and shows it outperforms or matches existing approaches on the noisy-or model.
Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any posterior without leveraging the structures of the generative models. In this paper, we propose Amortized Conjugate Posterior (ACP), a hybrid approach taking advantages of both types of approaches. Specifically, we use the classical methods to derive specific forms of posterior distributions and then learn the variational parameters using amortized inference. We study the effectiveness of the proposed approach on the noisy-or model and compare to both the classical and the modern approaches for approximate inference and parameter learning. Our results show that the proposed method outperforms or are at par with other approaches.