A Distribution Similarity Based Regularizer for Learning Bayesian Networks
This work addresses the challenge of improving generalization in probabilistic graphical models for domain-specific applications like wave propagation modeling, though it appears incremental as it builds on existing regularization concepts.
The authors tackled the problem of learning Bayesian networks by introducing a distribution similarity regularizer to exploit common information among conditional probability distributions, resulting in a method that successfully models wave propagation in inhomogeneous media where baseline methods fail.
Probabilistic graphical models compactly represent joint distributions by decomposing them into factors over subsets of random variables. In Bayesian networks, the factors are conditional probability distributions. For many problems, common information exists among those factors. Adding similarity restrictions can be viewed as imposing prior knowledge for model regularization. With proper restrictions, learned models usually generalize better. In this work, we study methods that exploit such high-level similarities to regularize the learning process and apply them to the task of modeling the wave propagation in inhomogeneous media. We propose a novel distribution-based penalization approach that encourages similar conditional probability distribution rather than force the parameters to be similar explicitly. We show in experiment that our proposed algorithm solves the modeling wave propagation problem, which other baseline methods are not able to solve.