Score and Information for Recursive Exponential Models with Incomplete Data
This work addresses the need for more sophisticated statistical modeling in Bayesian networks by integrating expert opinions, though it appears incremental as it builds on existing recursive graphical models.
The paper tackles the problem of incorporating imprecise expert knowledge into recursive exponential models for probabilistic expert systems, showing how to translate such knowledge into conjugate prior distributions and deriving score and observed information for both likelihood and posterior based on incomplete data.
Recursive graphical models usually underlie the statistical modelling concerning probabilistic expert systems based on Bayesian networks. This paper defines a version of these models, denoted as recursive exponential models, which have evolved by the desire to impose sophisticated domain knowledge onto local fragments of a model. Besides the structural knowledge, as specified by a given model, the statistical modelling may also include expert opinion about the values of parameters in the model. It is shown how to translate imprecise expert knowledge into approximately conjugate prior distributions. Based on possibly incomplete data, the score and the observed information are derived for these models. This accounts for both the traditional score and observed information, derived as derivatives of the log-likelihood, and the posterior score and observed information, derived as derivatives of the log-posterior distribution. Throughout the paper the specialization into recursive graphical models is accounted for by a simple example.