Probabilistic graphs using coupled random variables
This work addresses the problem of designing more interpretable and efficient neural networks for machine learning practitioners, though it appears incremental as it builds on existing probabilistic models with a specific generalization.
The paper tackled the trade-off between expressiveness and traceability in neural networks by introducing a probabilistic graph model using coupled random variables, which replaced thousands of linear correlation parameters with a single coupling parameter while achieving only a 3-4% reduction in classification and inference performance on the UCI MLR dataset.
Neural network design has utilized flexible nonlinear processes which can mimic biological systems, but has suffered from a lack of traceability in the resulting network. Graphical probabilistic models ground network design in probabilistic reasoning, but the restrictions reduce the expressive capability of each node making network designs complex. The ability to model coupled random variables using the calculus of nonextensive statistical mechanics provides a neural node design incorporating nonlinear coupling between input states while maintaining the rigor of probabilistic reasoning. A generalization of Bayes rule using the coupled product enables a single node to model correlation between hundreds of random variables. A coupled Markov random field is designed for the inferencing and classification of UCI's MLR 'Multiple Features Data Set' such that thousands of linear correlation parameters can be replaced with a single coupling parameter with just a (3%, 4%) percent reduction in (classification, inference) performance.