Sinusoidal Flow: A Fast Invertible Autoregressive Flow
This work addresses a bottleneck in normalizing flows for researchers and practitioners in machine learning, offering a novel method that improves efficiency without sacrificing performance.
The paper tackles the challenge of balancing expressiveness, fast inversion, and exact Jacobian determinant in normalizing flows by proposing Sinusoidal Flow, which achieves this balance and models complex distributions while generating realistic samples with stacked layers.
Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.