Variational Autoencoders for Sparse and Overdispersed Discrete Data
This addresses the challenge of overdispersion in discrete data for applications like text modeling and recommender systems, representing an incremental improvement over existing latent factor models.
The paper tackled the problem of modeling sparse, high-dimensional, and overdispersed discrete data, such as in text analysis and collaborative filtering, by proposing a variational autoencoder framework using negative-binomial distributions, which achieved significantly better performance compared to state-of-the-art baselines.
Many applications, such as text modelling, high-throughput sequencing, and recommender systems, require analysing sparse, high-dimensional, and overdispersed discrete (count-valued or binary) data. Although probabilistic matrix factorisation and linear/nonlinear latent factor models have enjoyed great success in modelling such data, many existing models may have inferior modelling performance due to the insufficient capability of modelling overdispersion in count-valued data and model misspecification in general. In this paper, we comprehensively study these issues and propose a variational autoencoder based framework that generates discrete data via negative-binomial distribution. We also examine the model's ability to capture properties, such as self- and cross-excitations in discrete data, which is critical for modelling overdispersion. We conduct extensive experiments on three important problems from discrete data analysis: text analysis, collaborative filtering, and multi-label learning. Compared with several state-of-the-art baselines, the proposed models achieve significantly better performance on the above problems.