Training Energy-Based Normalizing Flow with Score-Matching Objectives
This addresses a bottleneck in generative modeling for researchers and practitioners by making training more efficient and stable, though it is an incremental improvement over existing methods.
The paper tackles the computational inefficiency of training flow-based generative models by introducing energy-based normalizing flow (EBFlow), which bypasses Jacobian determinant computations using score-matching objectives, achieving significant speedup and outperforming prior methods in negative log-likelihood.
In this paper, we establish a connection between the parameterization of flow-based and energy-based generative models, and present a new flow-based modeling approach called energy-based normalizing flow (EBFlow). We demonstrate that by optimizing EBFlow with score-matching objectives, the computation of Jacobian determinants for linear transformations can be entirely bypassed. This feature enables the use of arbitrary linear layers in the construction of flow-based models without increasing the computational time complexity of each training iteration from $O(D^2L)$ to $O(D^3L)$ for an $L$-layered model that accepts $D$-dimensional inputs. This makes the training of EBFlow more efficient than the commonly-adopted maximum likelihood training method. In addition to the reduction in runtime, we enhance the training stability and empirical performance of EBFlow through a number of techniques developed based on our analysis of the score-matching methods. The experimental results demonstrate that our approach achieves a significant speedup compared to maximum likelihood estimation while outperforming prior methods with a noticeable margin in terms of negative log-likelihood (NLL).