Hellinger Multimodal Variational Autoencoders
This work addresses multimodal generative learning for researchers, offering an incremental improvement over existing methods like product or mixture of experts.
The paper tackled the problem of multimodal variational autoencoders by proposing HELVAE, which uses Hellinger pooling to improve latent representation expressiveness and generative performance, achieving better trade-offs between coherence and quality compared to state-of-the-art models.
Multimodal variational autoencoders (VAEs) are widely used for weakly supervised generative learning with multiple modalities. Predominant methods aggregate unimodal inference distributions using either a product of experts (PoE), a mixture of experts (MoE), or their combinations to approximate the joint posterior. In this work, we revisit multimodal inference through the lens of probabilistic opinion pooling, an optimization-based approach. We start from Hölder pooling with $α=0.5$, which corresponds to the unique symmetric member of the $α\text{-divergence}$ family, and derive a moment-matching approximation, termed Hellinger. We then leverage such an approximation to propose HELVAE, a multimodal VAE that avoids sub-sampling, yielding an efficient yet effective model that: (i) learns more expressive latent representations as additional modalities are observed; and (ii) empirically achieves better trade-offs between generative coherence and quality, outperforming state-of-the-art multimodal VAE models.