Integrating Probabilistic Rules into Neural Networks: A Stochastic EM Learning Algorithm
This work addresses the challenge of handling cycles, hidden variables, and uncertain evidence in probabilistic neural networks, though it appears incremental as it adapts an existing algorithm to a specific case.
The paper tackles the problem of integrating probabilistic rules into neural networks by adapting a stochastic EM algorithm, enabling the combination of probabilistic inference networks with unspecified associations in a sound manner.
The EM-algorithm is a general procedure to get maximum likelihood estimates if part of the observations on the variables of a network are missing. In this paper a stochastic version of the algorithm is adapted to probabilistic neural networks describing the associative dependency of variables. These networks have a probability distribution, which is a special case of the distribution generated by probabilistic inference networks. Hence both types of networks can be combined allowing to integrate probabilistic rules as well as unspecified associations in a sound way. The resulting network may have a number of interesting features including cycles of probabilistic rules, hidden 'unobservable' variables, and uncertain and contradictory evidence.