Fractal Generative Models
This work proposes a new paradigm in generative modeling, potentially impacting researchers and practitioners in machine learning by offering a novel architectural approach.
The paper tackles the problem of modularizing generative models by introducing fractal generative models that recursively use atomic generative modules, demonstrating strong performance in likelihood estimation and generation quality for pixel-by-pixel image generation.
Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models into atomic generative modules. Analogous to fractals in mathematics, our method constructs a new type of generative model by recursively invoking atomic generative modules, resulting in self-similar fractal architectures that we call fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic generative modules and examine it on the challenging task of pixel-by-pixel image generation, demonstrating strong performance in both likelihood estimation and generation quality. We hope this work could open a new paradigm in generative modeling and provide a fertile ground for future research. Code is available at https://github.com/LTH14/fractalgen.