Score Neural Operator: A Generative Model for Learning and Generalizing Across Multiple Probability Distributions
This addresses the problem of limited generalization in generative models for AI researchers, offering a novel approach that is not incremental but introduces a new method for a known bottleneck.
The paper tackles the limitation of existing generative models to learn only a single probability distribution by introducing the Score Neural Operator, which learns mappings from multiple distributions to their score functions, enabling generalization to unseen distributions and achieving strong performance on datasets like 2D Gaussian Mixture Models and 1024-dimensional MNIST double-digit datasets.
Most existing generative models are limited to learning a single probability distribution from the training data and cannot generalize to novel distributions for unseen data. An architecture that can generate samples from both trained datasets and unseen probability distributions would mark a significant breakthrough. Recently, score-based generative models have gained considerable attention for their comprehensive mode coverage and high-quality image synthesis, as they effectively learn an operator that maps a probability distribution to its corresponding score function. In this work, we introduce the $\emph{Score Neural Operator}$, which learns the mapping from multiple probability distributions to their score functions within a unified framework. We employ latent space techniques to facilitate the training of score matching, which tends to over-fit in the original image pixel space, thereby enhancing sample generation quality. Our trained Score Neural Operator demonstrates the ability to predict score functions of probability measures beyond the training space and exhibits strong generalization performance in both 2-dimensional Gaussian Mixture Models and 1024-dimensional MNIST double-digit datasets. Importantly, our approach offers significant potential for few-shot learning applications, where a single image from a new distribution can be leveraged to generate multiple distinct images from that distribution.