Cooperative neural networks (CoNN): Exploiting prior independence structure for improved classification
This work addresses the challenge of integrating domain-specific prior knowledge into deep learning models for text classification, representing an incremental improvement over existing approaches.
The authors tackled the problem of incorporating prior independence structure into neural networks for improved classification, achieving a 23% reduction in error on the MultiSent dataset compared to state-of-the-art methods.
We propose a new approach, called cooperative neural networks (CoNN), which uses a set of cooperatively trained neural networks to capture latent representations that exploit prior given independence structure. The model is more flexible than traditional graphical models based on exponential family distributions, but incorporates more domain specific prior structure than traditional deep networks or variational autoencoders. The framework is very general and can be used to exploit the independence structure of any graphical model. We illustrate the technique by showing that we can transfer the independence structure of the popular Latent Dirichlet Allocation (LDA) model to a cooperative neural network, CoNN-sLDA. Empirical evaluation of CoNN-sLDA on supervised text classification tasks demonstrates that the theoretical advantages of prior independence structure can be realized in practice -we demonstrate a 23\% reduction in error on the challenging MultiSent data set compared to state-of-the-art.