Hidden Tree Markov Networks: Deep and Wide Learning for Structured Data
This addresses the challenge of efficiently modeling structured data for applications in fields like natural language processing, though it appears incremental as it builds on existing probabilistic and neural methods.
The paper tackles the problem of learning from structured data like trees by introducing the Hidden Tree Markov Network (HTN), a hybrid model combining generative models and neural networks, and shows it outperforms state-of-the-art syntactic and generative kernels in experiments.
The paper introduces the Hidden Tree Markov Network (HTN), a neuro-probabilistic hybrid fusing the representation power of generative models for trees with the incremental and discriminative learning capabilities of neural networks. We put forward a modular architecture in which multiple generative models of limited complexity are trained to learn structural feature detectors whose outputs are then combined and integrated by neural layers at a later stage. In this respect, the model is both deep, thanks to the unfolding of the generative models on the input structures, as well as wide, given the potentially large number of generative modules that can be trained in parallel. Experimental results show that the proposed approach can outperform state-of-the-art syntactic kernels as well as generative kernels built on the same probabilistic model as the HTN.