A New Distribution on the Simplex with Auto-Encoding Applications
This work addresses the need for efficient and interpretable probability distributions in variational Bayesian modeling for machine learning, particularly in semi-supervised auto-encoding, though it is incremental as it builds on existing distributions like the Dirichlet.
The authors introduced a new distribution for the simplex based on the Kumaraswamy distribution and an ordered stick-breaking process, which captures sparsity like the Dirichlet but has an exact closed-form reparameterization, making it suitable for deep variational Bayesian modeling. They demonstrated its utility in semi-supervised auto-encoding tasks, achieving competitive performance with simpler, explicit probability models and no adversarial training.
We construct a new distribution for the simplex using the Kumaraswamy distribution and an ordered stick-breaking process. We explore and develop the theoretical properties of this new distribution and prove that it exhibits symmetry under the same conditions as the well-known Dirichlet. Like the Dirichlet, the new distribution is adept at capturing sparsity but, unlike the Dirichlet, has an exact and closed form reparameterization--making it well suited for deep variational Bayesian modeling. We demonstrate the distribution's utility in a variety of semi-supervised auto-encoding tasks. In all cases, the resulting models achieve competitive performance commensurate with their simplicity, use of explicit probability models, and abstinence from adversarial training.