ELF: Exact-Lipschitz Based Universal Density Approximator Flow
This addresses computational inefficiency in normalizing flows for the broader machine learning community, though it appears incremental as it builds on existing flow methods.
The paper tackled the problem of normalizing flows being computationally expensive by introducing ELF, a one-dimensional one-layer network with closed-form Lipschitz constants, which achieved state-of-the-art performance on multiple large-scale datasets.
Normalizing flows have grown more popular over the last few years; however, they continue to be computationally expensive, making them difficult to be accepted into the broader machine learning community. In this paper, we introduce a simple one-dimensional one-layer network that has closed form Lipschitz constants; using this, we introduce a new Exact-Lipschitz Flow (ELF) that combines the ease of sampling from residual flows with the strong performance of autoregressive flows. Further, we show that ELF is provably a universal density approximator, more computationally and parameter efficient compared to a multitude of other flows, and achieves state-of-the-art performance on multiple large-scale datasets.