Non-Parametric Learning of Lifted Restricted Boltzmann Machines
This work addresses the problem of improving interpretability in relational data modeling for machine learning practitioners, though it appears incremental as it builds on existing RBM and gradient-boosting techniques.
The paper tackles discriminative learning of restricted Boltzmann machines for relational data by introducing a gradient-boosted method that simultaneously learns structure and parameters using relational regression trees, resulting in more interpretable models without loss in effectiveness.
We consider the problem of discriminatively learning restricted Boltzmann machines in the presence of relational data. Unlike previous approaches that employ a rule learner (for structure learning) and a weight learner (for parameter learning) sequentially, we develop a gradient-boosted approach that performs both simultaneously. Our approach learns a set of weak relational regression trees, whose paths from root to leaf are conjunctive clauses and represent the structure, and whose leaf values represent the parameters. When the learned relational regression trees are transformed into a lifted RBM, its hidden nodes are precisely the conjunctive clauses derived from the relational regression trees. This leads to a more interpretable and explainable model. Our empirical evaluations clearly demonstrate this aspect, while displaying no loss in effectiveness of the learned models.