Deep Exponential Families
This work provides a novel approach for improving predictions in latent variable models, benefiting researchers and practitioners in machine learning, though it appears incremental as it builds on existing neural network and exponential family concepts.
The authors tackled the problem of modeling hierarchical dependencies in latent variable models by introducing deep exponential families (DEFs), which combine deep neural network structures with exponential families, and demonstrated that DEFs achieve better predictive performance than state-of-the-art models on text and recommendation data.
We describe \textit{deep exponential families} (DEFs), a class of latent variable models that are inspired by the hidden structures used in deep neural networks. DEFs capture a hierarchy of dependencies between latent variables, and are easily generalized to many settings through exponential families. We perform inference using recent "black box" variational inference techniques. We then evaluate various DEFs on text and combine multiple DEFs into a model for pairwise recommendation data. In an extensive study, we show that going beyond one layer improves predictions for DEFs. We demonstrate that DEFs find interesting exploratory structure in large data sets, and give better predictive performance than state-of-the-art models.