Meta-probabilistic Modeling
This addresses the problem of model specification in probabilistic modeling for researchers and practitioners, though it appears incremental as it builds on existing meta-learning and VAE techniques.
The paper tackles the challenge of selecting well-specified probabilistic graphical models by proposing meta-probabilistic modeling (MPM), a meta-learning algorithm that learns generative model structure from multiple related datasets, resulting in adaptation to data and recovery of meaningful latent representations.
While probabilistic graphical models can discover latent structure in data, their effectiveness hinges on choosing well-specified models. Identifying such models is challenging in practice, often requiring iterative checking and revision through trial and error. To this end, we propose meta-probabilistic modeling (MPM), a meta-learning algorithm that learns generative model structure directly from multiple related datasets. MPM uses a hierarchical architecture where global model specifications are shared across datasets while local parameters remain dataset-specific. For learning and inference, we propose a tractable VAE-inspired surrogate objective, and optimize it through bi-level optimization: local variables are updated analytically via coordinate ascent, while global parameters are trained with gradient-based methods. We evaluate MPM on object-centric image modeling and sequential text modeling, demonstrating that it adapts generative models to data while recovering meaningful latent representations.