NanoFlow: Scalable Normalizing Flows with Sublinear Parameter Complexity
This work addresses the scalability issue in normalizing flows for researchers and practitioners in deep generative modeling, offering a parameter-efficient solution.
The paper tackled the problem of high parameter complexity in normalizing flows by introducing NanoFlow, a method that uses a single neural density estimator for multiple transformation stages, achieving significant sublinear parameter complexity in experiments on audio and image models.
Normalizing flows (NFs) have become a prominent method for deep generative models that allow for an analytic probability density estimation and efficient synthesis. However, a flow-based network is considered to be inefficient in parameter complexity because of reduced expressiveness of bijective mapping, which renders the models unfeasibly expensive in terms of parameters. We present an alternative parameterization scheme called NanoFlow, which uses a single neural density estimator to model multiple transformation stages. Hence, we propose an efficient parameter decomposition method and the concept of flow indication embedding, which are key missing components that enable density estimation from a single neural network. Experiments performed on audio and image models confirm that our method provides a new parameter-efficient solution for scalable NFs with significant sublinear parameter complexity.