Quasi-Autoregressive Residual (QuAR) Flows
This work addresses a bottleneck for practitioners in machine learning by making flow-based modeling more tractable and broadening its applicability, though it is incremental as it builds on existing residual flow methods.
The paper tackles the computational expense of residual flows in normalizing flows by introducing a Quasi-Autoregressive (QuAR) simplification, which dramatically reduces compute time and memory requirements while retaining many benefits.
Normalizing Flows are a powerful technique for learning and modeling probability distributions given samples from those distributions. The current state of the art results are built upon residual flows as these can model a larger hypothesis space than coupling layers. However, residual flows are extremely computationally expensive both to train and to use, which limits their applicability in practice. In this paper, we introduce a simplification to residual flows using a Quasi-Autoregressive (QuAR) approach. Compared to the standard residual flow approach, this simplification retains many of the benefits of residual flows while dramatically reducing the compute time and memory requirements, thus making flow-based modeling approaches far more tractable and broadening their potential applicability.